diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..a344e1fd3ea52d9fc0eb48e8fbc98baca948339c 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -17,6 +17,7 @@
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
+*.pdf filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
@@ -33,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000000000000000000000000000000000000..2c3796c32bff755ac50f4a36673a22994dd774f2
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,96 @@
+# Upstage Solar License
+
+# Preamble
+
+The 'Upstage Solar License' (hereinafter referred to as "this License") was established by Upstage Co., Ltd., incorporated under the laws of the Republic of Korea, to encourage the development of open-source software using Solar AI models, and is not affiliated with the Apache Software Foundation.
+
+This License basically adopts all provisions of the Apache License, Version 2.0 (hereinafter referred to as "Apache License 2.0"), including the principle of allowing commercial use, but prescribes minimum strategic conditions for the global expansion of AI technology and the sustainable development of the AI ecosystem.
+
+The key additional condition, as specified in Section 4(e), is that if you distribute a "Derivative AI Model" based on the "Work", you must specify the 'Solar' brand. This applies as an exception to Section 6 (Trademarks) of the Apache License 2.0.
+
+# TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1\. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
+
+"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
+
+2\. Grant of Copyright License.
+Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
+
+3\. Grant of Patent License.
+Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
+
+4\. Redistribution.
+You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
+
+(a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
+(b) You must cause any modified files to carry prominent notices stating that You changed the files; and
+(c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
+(d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
+(e) If You distribute or make available a Derivative Work that is an artificial intelligence model created, trained, fine-tuned, or otherwise improved using the Work (the "Derivative AI Model"), You must adhere to the following conditions:
+(i) The name of such Derivative AI Model must begin with "Solar" (e.g., "Solar-MyModel-v1"); and
+(ii) You must prominently display the phrase "Built with Solar" in any related websites, user interfaces, or documentation associated with the Derivative AI Model; and
+(iii) You must provide a copy of this License, including the original copyright notice and NOTICE file as included in the distribution of the Works, alongside the Derivative AI Model.
+
+You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
+
+5\. Submission of Contributions.
+Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
+
+6\. Trademarks.
+This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work, reproducing the content of the NOTICE file, and as explicitly required for attribution in Section 4(e) of this License.
+Any use of the “Solar” name under this License must not imply any sponsorship, endorsement, certification, or official relationship with the Licensor, nor mislead users into believing that a Derivative AI Model is an official product of the Licensor.
+
+7\. Disclaimer of Warranty.
+Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
+
+8\. Limitation of Liability.
+In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
+
+9\. Accepting Warranty or Additional Liability.
+While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+# APPENDIX: How to apply the Upstage Solar License to your work.
+
+To apply the Upstage Solar License to your work, attach the following boilerplate notice, with the fields enclosed by brackets replaced with your own identifying information. (Don't include the brackets\!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
+
+Copyright \[yyyy\] \[Upstage AI (or other copyright owner)\]
+
+Licensed under the Upstage Solar License (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ https://huggingface.co/Upstage/Solar-Open-100B/blob/main/LICENSE
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+This work is based on or derived from materials licensed under the
+Upstage Solar License.
+If you distribute or make available a Derivative AI Model (as defined
+in Section 4(e) of the License), your model name must begin with "Solar"
+and you must prominently display "Built with Solar" in associated
+documentation or interfaces.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..ca2f5920d8e4a335cd74967088bbf04588e8f4f2
--- /dev/null
+++ b/README.md
@@ -0,0 +1,120 @@
+---
+license: apache-2.0
+base_model: upstage/Solar-Open-100B
+tags:
+- quantization
+- mxfp4
+- amd
+- quark
+- rocm
+- quanto
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# Solar-Open-100B-MXFP4
+
+MXFP4 quantized version of [upstage/Solar-Open-100B](https://huggingface.co/upstage/Solar-Open-100B), quantized using [quanto](https://github.com/haanjack/quanto) with AMD Quark file-to-file quantization.
+
+## Quantization Details
+
+| Property | Value |
+|----------|-------|
+| Base Model | upstage/Solar-Open-100B |
+| Quantization Method | OCP MXFP4 (E2M1 values + E8M0 scales) |
+| Quantization Tool | quanto + AMD Quark 0.11.1 (file-to-file) |
+| Algorithm | RTN (Round-To-Nearest) |
+| Group Size | 32 |
+| Calibration Data | pileval (128 samples) |
+| Original Size | 192 GB |
+| Quantized Size | 53 GB |
+| Compression Ratio | 3.62x |
+
+## Benchmark Results
+
+### MMLU (5-shot)
+
+| Category | Baseline (BF16) | MXFP4 | Delta |
+|----------|:---------------:|:-----:|:-----:|
+| **Overall** | **77.58%** | **76.14%** | **-1.44%** |
+| Humanities | 71.09% | 70.10% | -0.99% |
+| Social Sciences | 86.90% | 85.12% | -1.78% |
+| STEM | 74.44% | 72.06% | -2.38% |
+| Other | 81.36% | 80.56% | -0.80% |
+
+### KMMLU (5-shot, Korean)
+
+| Category | Baseline (BF16) | MXFP4 | Delta |
+|----------|:---------------:|:-----:|:-----:|
+| **Overall** | **57.38%** | **57.03%** | **-0.35%** |
+| Applied Science | 53.28% | 53.08% | -0.20% |
+| HUMSS | 66.43% | 65.01% | -1.42% |
+| STEM | 58.13% | 58.17% | +0.04% |
+| Other | 56.64% | 56.29% | -0.35% |
+
+## Excluded Layers
+
+The following layers are excluded from quantization (kept in BF16) based on MXFP4-aware iterative sensitivity analysis with threshold 0.12, plus vLLM fused layer alignment:
+
+**Self-Attention (layers 0-2)**: q_proj, k_proj, v_proj — early layers are most sensitive to attention quantization.
+
+**Shared Expert MLP (layers 0-7)**: gate_proj, up_proj — early shared expert FFN layers show higher sensitivity. down_proj additionally excluded in layers 6-7 and 46-47.
+
+**MoE Router Gates**: All mlp.gate layers (topk router) — these are standard nn.Linear layers not compatible with MXFP4 packed inference.
+
+**Standard exclusions**: lm_head, embed_tokens, all norm layers.
+
+
+Full exclude list (33 layers)
+
+```
+lm_head, *embed*, *norm*, *.gate
+model.layers.0.self_attn.{q,k,v}_proj
+model.layers.0.mlp.shared_experts.{gate,up}_proj
+model.layers.1.self_attn.{q,k,v}_proj
+model.layers.1.mlp.shared_experts.{gate,up}_proj
+model.layers.2.self_attn.{q,k,v}_proj
+model.layers.2.mlp.shared_experts.{gate,up}_proj
+model.layers.3.mlp.shared_experts.{gate,up}_proj
+model.layers.4.mlp.shared_experts.{gate,up}_proj
+model.layers.5.mlp.shared_experts.{gate,up}_proj
+model.layers.6.mlp.shared_experts.{gate,up,down}_proj
+model.layers.7.mlp.shared_experts.{gate,up,down}_proj
+model.layers.46.mlp.shared_experts.down_proj
+model.layers.47.mlp.shared_experts.down_proj
+```
+
+
+## Hardware
+
+- Quantized on: AMD MI355 (gfx950), 288 GB VRAM
+- Tested with: vLLM v0.18.1 (ROCm), TP=1
+- Compatible with: vLLM with Quark quantization support (quant_method: "quark")
+
+## Usage
+
+### vLLM
+
+```bash
+vllm serve haanjack/Solar-Open-100B-MXFP4 \
+ --trust-remote-code \
+ --tensor-parallel-size 1 \
+ --enforce-eager \
+ --max-model-len 4096
+```
+
+### Quantization Reproduction
+
+```python
+from quanto import UnifiedQuantizer, UnifiedConfig
+
+config = UnifiedConfig(
+ model_path="upstage/Solar-Open-100B",
+ output_dir="./Solar-Open-100B-MXFP4",
+ precision="mxfp4",
+ sensitivity_analysis=True,
+ sensitivity_threshold=0.12,
+ num_calib_samples=128,
+)
+UnifiedQuantizer(config).run()
+```
diff --git a/Solar-Open-100B.png b/Solar-Open-100B.png
new file mode 100644
index 0000000000000000000000000000000000000000..68fdccb000b1d6e615bab5f808a2a0ff5af9bbc6
Binary files /dev/null and b/Solar-Open-100B.png differ
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..13268c1a841b2dbe113bdc6dc2c11c3200d4067d
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1,156 @@
+{#- ======== Template Parameters ======== #}
+{%- set add_generation_prompt = add_generation_prompt if add_generation_prompt is defined else true %}
+{%- set default_system_prompt = default_system_prompt if default_system_prompt is defined else true %}
+{%- set reasoning_effort = reasoning_effort if reasoning_effort is defined else "high" %}
+{%- set think_render_option = think_render_option if think_render_option is defined else "lastthink" %}
+
+{#- ======== System Block State ======== #}
+{%- set sys_ns = namespace(is_first_block=true) -%}
+
+{#- ======== Find last user message index ======== #}
+{%- set last_user_idx = namespace(value=-1) -%}
+{%- for message in messages -%}
+ {%- if message.role == 'user' -%}
+ {%- set last_user_idx.value = loop.index0 -%}
+ {%- endif -%}
+{%- endfor -%}
+
+{#- ======== System messages renderers ======== #}
+{%- macro render_system_message(user_system_messages) %}
+ {%- if default_system_prompt %}
+ {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %}
+ {%- set sys_ns.is_first_block = false %}
+ {{- "## Provider System Prompt\n\nYou are Solar Open 100B, a large language model trained by Upstage AI, a Korean startup. Your knowledge cutoff is 2025-07. The current date is " + strftime_now("%Y-%m-%d") + "." }}
+ {%- endif -%}
+ {%- if user_system_messages %}
+ {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %}
+ {%- set sys_ns.is_first_block = false %}
+ {{- "## System Prompt" }}
+ {%- for system_message in user_system_messages %}
+ {{- "\n\n" }}
+ {{- system_message }}
+ {%- endfor %}
+ {%- endif -%}
+{%- endmacro %}
+
+{%- macro render_tool_instruction(tools) %}
+ {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %}
+ {%- set sys_ns.is_first_block = false %}
+ {{- "## Tools\n\n### Tool Call Instruction" }}
+ {{- "\nYou may invoke one or more tools to assist with the user's query. Available tools are provided in JSON Schema format: <|tools:begin|><|tool:begin|><|tool:end|>...<|tools:end|>\n" }}
+ {{- "\n### Available Tools\n" }}
+ {{- "<|tools:begin|>" }}
+ {%- for tool in tools %}
+ {{- "<|tool:begin|>" }}
+ {{- tool.function | tojson }}
+ {{- "<|tool:end|>" }}
+ {%- endfor %}
+ {{- "<|tools:end|>\n" }}
+ {{- "\n### Tool Call Format\n" }}
+ {{- "For each tool call, return a JSON object with the following structure, enclosed within <|tool_call:begin|> and <|tool_call:end|> tags: \n<|tool_call:begin|><|tool_call:name|><|tool_call:args|><|tool_call:end|>\n" }}
+ {{- "- The must be a randomly generated string consisting of 10 lowercase letters (a-z) and/or digits (0-9) (e.g., a1b2c3d4e5)\n" }}
+ {{- "\n### Tool Response Format\n" }}
+ {{- "Each tool is responded by `tool` with the following structure:\n<|tool_response:id|><|tool_response:name|><|tool_response:result|><|tool_response:end|>\n" }}
+ {{- "- Ensure the matches the corresponding tool call" -}}
+{%- endmacro %}
+
+{%- macro render_json_response_format_instruction(response_format) %}
+ {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %}
+ {%- set sys_ns.is_first_block = false %}
+ {{- "## Output Format Constraint" }}
+ {{- "\n\nYour final response should follow the JSON schema: \n[Start of schema]" }}
+ {{- response_format }}
+ {{- "\n[End of schema]\nPlease ensure your answers adhere to this format and do not contain any unnecessary text." }}
+{%- endmacro %}
+
+{%- macro get_tool_name(messages, tool_call_id) %}
+ {%- for msg in messages -%}
+ {%- if msg.role == 'assistant' and msg.tool_calls -%}
+ {%- for tool_call in msg.tool_calls -%}
+ {%- if tool_call.id == tool_call_id -%}
+ {{- tool_call.function.name }}
+ {%- endif -%}
+ {%- endfor -%}
+ {%- endif -%}
+ {%- endfor -%}
+{%- endmacro %}
+
+{%- macro render_tool_arguments(tool_arguments) %}
+ {%- if tool_arguments is mapping -%}
+ {{- tool_arguments | tojson }}
+ {%- else -%}
+ {{- tool_arguments }}
+ {%- endif -%}
+{%- endmacro %}
+
+{#- ======== Render system message ======== #}
+{%- set ns = namespace(system_messages=[]) -%}
+{%- for message in messages -%}
+ {%- if message.role == 'system' -%}
+ {%- set ns.system_messages = ns.system_messages + [message.content] -%}
+ {%- endif -%}
+{%- endfor -%}
+
+{%- if ns.system_messages or default_system_prompt or tools or response_format -%}
+ {{- "<|begin|>system<|content|>" }}
+ {{- render_system_message(ns.system_messages) }}
+ {%- if tools -%}
+ {{- render_tool_instruction(tools) }}
+ {%- endif %}
+ {%- if response_format -%}
+ {{- render_json_response_format_instruction(response_format) }}
+ {%- endif %}
+ {{- "<|end|>" }}
+{%- endif -%}
+
+{#- ======== Render main messages ======== #}
+{%- for message in messages -%}
+ {%- if message.role == 'user' -%}
+ {{- "<|begin|>user<|content|>" + message.content + "<|end|>" }}
+ {%- elif message.role == 'tool' -%}
+ {%- set prev_is_tool = loop.index0 > 0 and messages[loop.index0 - 1].role == 'tool' -%}
+ {%- set next_is_tool = loop.index0 < (messages | length - 1) and messages[loop.index0 + 1].role == 'tool' -%}
+ {%- if not prev_is_tool -%}
+ {{- "<|begin|>tool<|tool_response|>" }}
+ {%- endif -%}
+ {{- "<|tool_response:begin|>" + message.tool_call_id + "<|tool_response:name|>" }}
+ {{- get_tool_name(messages, message.tool_call_id) }}
+ {{- "<|tool_response:result|>" }}
+ {{- message.content }}
+ {{- "<|tool_response:end|>" }}
+ {%- if not next_is_tool -%}
+ {{- "<|end|>" }}
+ {%- endif -%}
+ {%- elif message.role == 'assistant' -%}
+ {#- ======== Assistant Thinking ======== #}
+ {%- if think_render_option == "all" -%}
+ {%- if message.reasoning -%}
+ {{- "<|begin|>assistant<|think|>" + message.reasoning + "<|end|>" }}
+ {%- endif -%}
+ {%- elif think_render_option == "lastthink" -%}
+ {%- if message.reasoning and loop.index0 > last_user_idx.value -%}
+ {{- "<|begin|>assistant<|think|>" + message.reasoning + "<|end|>" }}
+ {%- endif -%}
+ {%- endif -%}
+
+ {#- ======== Assistant Messages ======== #}
+ {%- if message.tool_calls -%}
+ {{- "<|begin|>assistant<|tool_calls|>" }}
+ {%- for tool_call in message.tool_calls -%}
+ {{- "<|tool_call:begin|>" + tool_call.id +"<|tool_call:name|>" + tool_call.function.name + "<|tool_call:args|>" }}
+ {{- render_tool_arguments(tool_call.function.arguments) }}
+ {{- "<|tool_call:end|>" }}
+ {%- endfor -%}
+ {{- "<|calls|>" }}
+ {%- else -%}
+ {{- "<|begin|>assistant<|content|>" + message.content + "<|end|>" }}
+ {%- endif -%}
+ {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+ {%- if reasoning_effort in ["low", "minimal"] -%}
+ {{- "<|begin|>assistant<|think|><|end|>" }}
+ {%- endif -%}
+ {{- "<|begin|>assistant" }}
+{%- endif -%}
diff --git a/config.json b/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..1ddccb684f32e52c8e9c7a6eb6d3717594eb609d
--- /dev/null
+++ b/config.json
@@ -0,0 +1,169 @@
+{
+ "model_type": "solar_open",
+ "architectures": [
+ "SolarOpenForCausalLM"
+ ],
+ "pad_token_id": 2,
+ "bos_token_id": 1,
+ "eos_token_id": 2,
+ "partial_rotary_factor": 1.0,
+ "hidden_size": 4096,
+ "num_hidden_layers": 48,
+ "num_attention_heads": 64,
+ "head_dim": 128,
+ "num_key_value_heads": 8,
+ "vocab_size": 196608,
+ "intermediate_size": 10240,
+ "moe_intermediate_size": 1280,
+ "rms_norm_eps": 1e-05,
+ "rope_theta": 1000000,
+ "max_position_embeddings": 131072,
+ "n_routed_experts": 128,
+ "n_shared_experts": 1,
+ "norm_topk_prob": true,
+ "routed_scaling_factor": 1.0,
+ "num_experts_per_tok": 8,
+ "tie_word_embeddings": false,
+ "torch_dtype": "bfloat16",
+ "first_k_dense_replace": 0,
+ "use_cache": true,
+ "transformers_version": "4.57.1",
+ "rope_scaling": {
+ "type": "yarn",
+ "factor": 2.0,
+ "original_max_position_embeddings": 65536
+ },
+ "quantization_config": {
+ "global_quant_config": {
+ "input_tensors": {
+ "dtype": "fp4",
+ "is_dynamic": true,
+ "qscheme": "per_group",
+ "ch_axis": -1,
+ "group_size": 32,
+ "symmetric": null,
+ "round_method": "half_even",
+ "scale_type": "float",
+ "scale_format": "e8m0",
+ "scale_calculation_mode": "even",
+ "mx_element_dtype": null,
+ "observer_cls": "PerBlockMXObserver",
+ "is_scale_quant": false
+ },
+ "output_tensors": null,
+ "weight": {
+ "dtype": "fp4",
+ "is_dynamic": false,
+ "qscheme": "per_group",
+ "ch_axis": -1,
+ "group_size": 32,
+ "symmetric": null,
+ "round_method": "half_even",
+ "scale_type": "float",
+ "scale_format": "e8m0",
+ "scale_calculation_mode": "even",
+ "mx_element_dtype": null,
+ "observer_cls": "PerBlockMXObserver",
+ "is_scale_quant": false
+ },
+ "bias": null,
+ "target_device": null
+ },
+ "exclude": [
+ "lm_head",
+ "model.layers.0.mlp.gate",
+ "model.layers.0.mlp.shared_experts.gate_proj",
+ "model.layers.0.mlp.shared_experts.up_proj",
+ "model.layers.0.self_attn.k_proj",
+ "model.layers.0.self_attn.q_proj",
+ "model.layers.0.self_attn.v_proj",
+ "model.layers.1.mlp.gate",
+ "model.layers.1.mlp.shared_experts.gate_proj",
+ "model.layers.1.mlp.shared_experts.up_proj",
+ "model.layers.1.self_attn.k_proj",
+ "model.layers.1.self_attn.q_proj",
+ "model.layers.1.self_attn.v_proj",
+ "model.layers.10.mlp.gate",
+ "model.layers.11.mlp.gate",
+ "model.layers.12.mlp.gate",
+ "model.layers.13.mlp.gate",
+ "model.layers.14.mlp.gate",
+ "model.layers.15.mlp.gate",
+ "model.layers.16.mlp.gate",
+ "model.layers.17.mlp.gate",
+ "model.layers.18.mlp.gate",
+ "model.layers.19.mlp.gate",
+ "model.layers.2.mlp.gate",
+ "model.layers.2.mlp.shared_experts.gate_proj",
+ "model.layers.2.mlp.shared_experts.up_proj",
+ "model.layers.2.self_attn.k_proj",
+ "model.layers.2.self_attn.q_proj",
+ "model.layers.2.self_attn.v_proj",
+ "model.layers.20.mlp.gate",
+ "model.layers.21.mlp.gate",
+ "model.layers.22.mlp.gate",
+ "model.layers.23.mlp.gate",
+ "model.layers.24.mlp.gate",
+ "model.layers.25.mlp.gate",
+ "model.layers.26.mlp.gate",
+ "model.layers.27.mlp.gate",
+ "model.layers.28.mlp.gate",
+ "model.layers.29.mlp.gate",
+ "model.layers.3.mlp.gate",
+ "model.layers.3.mlp.shared_experts.gate_proj",
+ "model.layers.3.mlp.shared_experts.up_proj",
+ "model.layers.30.mlp.gate",
+ "model.layers.31.mlp.gate",
+ "model.layers.32.mlp.gate",
+ "model.layers.33.mlp.gate",
+ "model.layers.34.mlp.gate",
+ "model.layers.35.mlp.gate",
+ "model.layers.36.mlp.gate",
+ "model.layers.37.mlp.gate",
+ "model.layers.38.mlp.gate",
+ "model.layers.39.mlp.gate",
+ "model.layers.4.mlp.gate",
+ "model.layers.4.mlp.shared_experts.gate_proj",
+ "model.layers.4.mlp.shared_experts.up_proj",
+ "model.layers.40.mlp.gate",
+ "model.layers.41.mlp.gate",
+ "model.layers.42.mlp.gate",
+ "model.layers.43.mlp.gate",
+ "model.layers.44.mlp.gate",
+ "model.layers.45.mlp.gate",
+ "model.layers.46.mlp.gate",
+ "model.layers.46.mlp.shared_experts.down_proj",
+ "model.layers.47.mlp.gate",
+ "model.layers.47.mlp.shared_experts.down_proj",
+ "model.layers.5.mlp.gate",
+ "model.layers.5.mlp.shared_experts.gate_proj",
+ "model.layers.5.mlp.shared_experts.up_proj",
+ "model.layers.6.mlp.gate",
+ "model.layers.6.mlp.shared_experts.down_proj",
+ "model.layers.6.mlp.shared_experts.gate_proj",
+ "model.layers.6.mlp.shared_experts.up_proj",
+ "model.layers.7.mlp.gate",
+ "model.layers.7.mlp.shared_experts.down_proj",
+ "model.layers.7.mlp.shared_experts.gate_proj",
+ "model.layers.7.mlp.shared_experts.up_proj",
+ "model.layers.8.mlp.gate",
+ "model.layers.9.mlp.gate"
+ ],
+ "algo_config": null,
+ "softmax_quant_spec": null,
+ "quant_method": "quark",
+ "layer_type_quant_config": {},
+ "layer_quant_config": {},
+ "kv_cache_quant_config": {},
+ "kv_cache_post_rope": false,
+ "quant_mode": "eager_mode",
+ "version": "0.11.1",
+ "export": {
+ "kv_cache_group": [],
+ "min_kv_scale": 0.0,
+ "pack_method": "reorder",
+ "weight_format": "real_quantized",
+ "weight_merge_groups": null
+ }
+ }
+}
\ No newline at end of file
diff --git a/generation_config.json b/generation_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..38bf7ecef7a70fcb0e81fdcfb5c74af94e68a1be
--- /dev/null
+++ b/generation_config.json
@@ -0,0 +1,14 @@
+{
+ "_from_model_config": true,
+ "bos_token_id": 1,
+ "eos_token_id": [
+ 2,
+ 24,
+ 25
+ ],
+ "pad_token_id": 2,
+ "transformers_version": "4.57.3",
+ "do_sample": true,
+ "temperature": 0.8,
+ "top_p": 0.95
+}
\ No newline at end of file
diff --git a/model-00001-of-00042.safetensors b/model-00001-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b50c77598b7141c61f7c99f002b1a1e7d9b631e4
--- /dev/null
+++ b/model-00001-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6594589b2bafbd66713744ba49e00273322dbdb5e635d2fa967843eb476bc363
+size 2570268192
diff --git a/model-00002-of-00042.safetensors b/model-00002-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..65fc75b620ecd29eecba82180c3256094005973c
--- /dev/null
+++ b/model-00002-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a42bb4309e991e189eaa0376dd9bfcc91e69661e2699db975baee3c53b8faded
+size 1405092432
diff --git a/model-00003-of-00042.safetensors b/model-00003-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4dbe2c6ac156e25ba56194ba555678659f4b6bd
--- /dev/null
+++ b/model-00003-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc78b828c5dd690bfb7022174e79ff92b2026e5d2ae61624477a27a020f15e4
+size 1419884920
diff --git a/model-00004-of-00042.safetensors b/model-00004-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4a8b402131281c5dbc644be3d9886b2dab27c74
--- /dev/null
+++ b/model-00004-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d13e3d6c92d5239e62a3190ae9b682d2ec86fd824644d348c9ec03853352405
+size 1343488576
diff --git a/model-00005-of-00042.safetensors b/model-00005-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3b5ea90d42baca5c0625526d2f68948f9696f15a
--- /dev/null
+++ b/model-00005-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bd586209025066fb78067fcac99b48364b5642c5d1187e79ce6ebef3aeb7131
+size 1343488576
diff --git a/model-00006-of-00042.safetensors b/model-00006-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..45ca173c7b25c422e2179e1e4daf330d190999e1
--- /dev/null
+++ b/model-00006-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b84b17605295473f55a55f7d336abe170d726b07b4830c3997420641e2a1f0d
+size 1343488576
diff --git a/model-00007-of-00042.safetensors b/model-00007-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..cca26d84f6939a96f8f45dcf535496b580db93d8
--- /dev/null
+++ b/model-00007-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01b27a2186c1d4e94eeaec358f5388801e44562ef2ab4aa92bf2055716c0bd34
+size 1351189072
diff --git a/model-00008-of-00042.safetensors b/model-00008-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..42a7b856503b7286bfb1f6da3445f7df2c7ffeef
--- /dev/null
+++ b/model-00008-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3855f11c6c291598f2d12486e96945abd4ee653463ddedd866b358e5a6213e85
+size 1350580928
diff --git a/model-00009-of-00042.safetensors b/model-00009-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6def17e7f4221d604b7ab6631e27a6517182caf5
--- /dev/null
+++ b/model-00009-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:121c4d336b863528419bfbac5c5dcc0a03540457ea5dbf05d6c15220d44d26a2
+size 1328088064
diff --git a/model-00010-of-00042.safetensors b/model-00010-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a76e02b97411d2c07dcb39ad2eb344f69dd1212a
--- /dev/null
+++ b/model-00010-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deff23fde26f8591f2b8704e7d4fea1a4a5f484b109a4ce25561b06feb7af542
+size 1328088752
diff --git a/model-00011-of-00042.safetensors b/model-00011-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0f8230ef1d4f2e4588cb9efc9013240dcabcc04d
--- /dev/null
+++ b/model-00011-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db879d56c997b32c3d159372c653a53e3cd50f833c00ecf4012fbcd72ad309ff
+size 1328088752
diff --git a/model-00012-of-00042.safetensors b/model-00012-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..82f090961b92da529cbf90fd8420da7bbcbe6151
--- /dev/null
+++ b/model-00012-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fa8078375892686cba038915e20710c2212641f1143d092236c7e44078afc27
+size 1328088808
diff --git a/model-00013-of-00042.safetensors b/model-00013-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2cab10977e25b63a8c88072c719faa5a6dacbd89
--- /dev/null
+++ b/model-00013-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aba0ed9ab95e6dd7e8c7bd06903f37af87938c0989bd13d3e694f32b3831da8
+size 1327480800
diff --git a/model-00014-of-00042.safetensors b/model-00014-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..375f2a845b031305d8923eaf7a4fcd56f72c5061
--- /dev/null
+++ b/model-00014-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32a4093c06111a41d0bae176d40aa24c136dfc9b0d400dbff1de3f8d6a315374
+size 1328088696
diff --git a/model-00015-of-00042.safetensors b/model-00015-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..87d3704d57c9a59549de84c38432b41dd879610d
--- /dev/null
+++ b/model-00015-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88d2c47c79ff4378421afaf9ae03b8127221418cd89e3b89fdcfa09c8c1f49cb
+size 1328088752
diff --git a/model-00016-of-00042.safetensors b/model-00016-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..4f858f1292c106deb9b16876852e899fb7f94267
--- /dev/null
+++ b/model-00016-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66f019dcdcec9658df78dfd97b696d2ff1e7f969a2afd004051836d4f46efc9b
+size 1328088752
diff --git a/model-00017-of-00042.safetensors b/model-00017-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..eebaf3302264eacf6a8e51c1ec33c492a90ef088
--- /dev/null
+++ b/model-00017-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a17e720629be2ccff3f315b4f0257793edbc7cae5e67e3e2969b0a029f64517
+size 1328088752
diff --git a/model-00018-of-00042.safetensors b/model-00018-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9f03664088e62d3d6c66c9e97cf706e04e1c7be3
--- /dev/null
+++ b/model-00018-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9bc46c8463b9ff9b04255ac357fa2b71d1a1abadbddaaf84e0d2d3e927b63a5
+size 1328088896
diff --git a/model-00019-of-00042.safetensors b/model-00019-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..11b27da1bd859e1f9d431ae866ca2eb46d838494
--- /dev/null
+++ b/model-00019-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c2b83e70e358243a20da370508ba4644251d9d4c5964df5a04296c483c8a6be
+size 1327480648
diff --git a/model-00020-of-00042.safetensors b/model-00020-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e8c52fff1fa366eb535a78ccc76010e751abbf8b
--- /dev/null
+++ b/model-00020-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9c7c45c471d989107555ef7a3274b475e0cbf72774f07d8a692e5bb6d40d364
+size 1328088752
diff --git a/model-00021-of-00042.safetensors b/model-00021-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b41faa28e64a6f0f002cdeac406c34b0011d9061
--- /dev/null
+++ b/model-00021-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a2b63cade87caf4a2703a69880eac5b7c267eb26108f570273a8ebd7bf0f770
+size 1328088752
diff --git a/model-00022-of-00042.safetensors b/model-00022-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..1b2d0788c5edd65bffa556e3818e0fd5199def7b
--- /dev/null
+++ b/model-00022-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8df43c9257ed6abc917a8bb4b57642629b08e3357e5bbf9afebabfcaf795121
+size 1328088752
diff --git a/model-00023-of-00042.safetensors b/model-00023-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..d73fb6888530855fd38cb692ce1fbd8307e1148b
--- /dev/null
+++ b/model-00023-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8492b162b9f4db47195eaef212d16af7357996662cc369feb954f51f8afc9c7
+size 1328088848
diff --git a/model-00024-of-00042.safetensors b/model-00024-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5c97d6f0d0be12be64efd40faab67aa3c49c72e9
--- /dev/null
+++ b/model-00024-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfa2b1ea14d322206375df6f7a5accc36169871fa42830642cb58542a694ce71
+size 1327480720
diff --git a/model-00025-of-00042.safetensors b/model-00025-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a0e05d2ff7b50bc53810be535a872c49157d06f5
--- /dev/null
+++ b/model-00025-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e93d05af145daa016b4225c54c0849354cccffb91bcfffbb019e4e580f67c97
+size 1328088736
diff --git a/model-00026-of-00042.safetensors b/model-00026-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..0f08fdd7a648ab7a87bf093195f745d63faa3b49
--- /dev/null
+++ b/model-00026-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ece777d5a0eb8d1c4941c57d4800265e209b9d32ffb6d3e68ee006a89959e8a
+size 1328088752
diff --git a/model-00027-of-00042.safetensors b/model-00027-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..029545efba71037206200309c60d70daff67c062
--- /dev/null
+++ b/model-00027-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8dc17fe7f9ea869a9070f42c1749effab65fc7ec0fafc93c6dfd10aed19c768a
+size 1328088752
diff --git a/model-00028-of-00042.safetensors b/model-00028-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3e018878e37c1b7e21dc5459971b0c0e3dd73a00
--- /dev/null
+++ b/model-00028-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1645e192721434747409e37977a9e003f144a6483dfa8f884920d19da64eb2ad
+size 1328088792
diff --git a/model-00029-of-00042.safetensors b/model-00029-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..f0621220e7a4978ba4cf724424272fe0849b5577
--- /dev/null
+++ b/model-00029-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:019f0c9f644d5e06239761078d9ceee79a6ae029d2aacf4c271289c798ea108d
+size 1320796912
diff --git a/model-00030-of-00042.safetensors b/model-00030-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..295f9fcc7a5ac5a8f57f25691e2a6ccebf425661
--- /dev/null
+++ b/model-00030-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb471f0988652c6a5791fd9ec9b5fc5866898703af22dd93a009c336294703ab
+size 1326416000
diff --git a/model-00031-of-00042.safetensors b/model-00031-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..163461362605115ab9f74f46a42409a09f3aedd3
--- /dev/null
+++ b/model-00031-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06f347596444dd08883a1aadb28712107bc945f46829098a30d6129e17690d18
+size 1328088752
diff --git a/model-00032-of-00042.safetensors b/model-00032-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b7197032dd841969ffe51b6f05e3a6ab11a35b02
--- /dev/null
+++ b/model-00032-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a8f1d424fc1362d9a94223d3c931b6c6fbd40d185d261a51c213d1195de7fac
+size 1328088752
diff --git a/model-00033-of-00042.safetensors b/model-00033-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..467201a7b3975f26955ac1e9f59f98d4be846589
--- /dev/null
+++ b/model-00033-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49bdb7866e4fb80270cadc68474a802df6990f83615b1eea1dd36682a2f45bc9
+size 1328088752
diff --git a/model-00034-of-00042.safetensors b/model-00034-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..cb5d24bc691498736cd8dfea05c9508e783b1598
--- /dev/null
+++ b/model-00034-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88b0f305173cce6c4af8c739144ea6b5bef42005667a006c416f778832fb70b0
+size 1328088888
diff --git a/model-00035-of-00042.safetensors b/model-00035-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..047da79df7b04760e0ce20f1112834ee2728ad97
--- /dev/null
+++ b/model-00035-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0039e26d9aae9381af44b8722085c9ea92135c0daec5fb69ebb71348688d3aa3
+size 1327480672
diff --git a/model-00036-of-00042.safetensors b/model-00036-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9c4e2204a18fd6507c98541a1915a90f49ffdb6e
--- /dev/null
+++ b/model-00036-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:894419f52a4dca42522e5fb2a3182e79c1278bf4f4789e7041a5ad93883a3897
+size 1328088752
diff --git a/model-00037-of-00042.safetensors b/model-00037-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a78a6c8e3af23befae5df1a2d836776f352725e9
--- /dev/null
+++ b/model-00037-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f94010ad07bea3acd2f474698096bce02fdeaf7ed19fa8fd238e0a8403271b1
+size 1328088752
diff --git a/model-00038-of-00042.safetensors b/model-00038-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e00fbf80f681e2728f08e61afa548faa2398277d
--- /dev/null
+++ b/model-00038-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb9fc33cf6bae31fde0ee27e208c1e2adfb9335992e99cad1f58eac96d139bd4
+size 1328088752
diff --git a/model-00039-of-00042.safetensors b/model-00039-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8b2c3508b536d4b0a1ce2992e37a3361e99d0a91
--- /dev/null
+++ b/model-00039-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:128f312582599ddfb75b3ddd9a3096e7e6ff74bcf1c8688d055748ac54f3e58f
+size 1328088832
diff --git a/model-00040-of-00042.safetensors b/model-00040-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..8ffb54e520e2dec9b0663e0703950da6a9185719
--- /dev/null
+++ b/model-00040-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5dfdba17d01d85dc0949b7d1dcb85be30ac46e3b64791178d425f5f5234c58d6
+size 1335181144
diff --git a/model-00041-of-00042.safetensors b/model-00041-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..9e789177dcde95a7c5a944a1d1e5bdfcf73ec222
--- /dev/null
+++ b/model-00041-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df6bc7afaf91c37f3772f3587a2a0b576d6373892f13c865d7b5e4fb727f444a
+size 1047776208
diff --git a/model-00042-of-00042.safetensors b/model-00042-of-00042.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ec96d93eaa05cee30bdfa61202c9fa84b72c6593
--- /dev/null
+++ b/model-00042-of-00042.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86a02fd21736b9c1fa04d08707182535cadf2a423875eff52b6601b5978d230b
+size 1610612832
diff --git a/model.safetensors.index.json b/model.safetensors.index.json
new file mode 100644
index 0000000000000000000000000000000000000000..ae1608b4a9443542b61ea7dd884396dbaac0f6ba
--- /dev/null
+++ b/model.safetensors.index.json
@@ -0,0 +1,37709 @@
+{
+ "metadata": {
+ "total_size": 57280405800
+ },
+ "weight_map": {
+ "model.layers.4.mlp.gate.e_score_correction_bias": "model-00005-of-00042.safetensors",
+ "model.layers.4.input_layernorm.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.100.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.101.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.102.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.103.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.104.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.105.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.106.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.107.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.108.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.109.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.110.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.111.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.112.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.113.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.114.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.115.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.116.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.117.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.118.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.119.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.120.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.121.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.122.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.123.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.124.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.125.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.126.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.127.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.44.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.45.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.46.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.47.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.48.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.49.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.50.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.51.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.52.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.53.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.54.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.55.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.56.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.57.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.58.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.59.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.60.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.61.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.62.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.63.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.64.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.65.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.66.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.67.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.68.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.69.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.70.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.71.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.72.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.73.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.74.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.75.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.76.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.77.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.78.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.79.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.80.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.81.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.82.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.83.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.84.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.85.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.86.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.87.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.88.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.89.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.90.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.91.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.92.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.93.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.94.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.95.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.96.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.97.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.98.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.experts.99.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.gate.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.shared_experts.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.shared_experts.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.shared_experts.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.mlp.shared_experts.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.4.post_attention_layernorm.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.0.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.1.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.10.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.11.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.12.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.13.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.14.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.15.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.16.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.17.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.18.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.19.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.2.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.20.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.21.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.22.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.23.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.24.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.25.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.26.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.27.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.28.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.29.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.3.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.30.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.31.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.32.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.33.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.34.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.35.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.36.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.37.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.38.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.39.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.4.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.40.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.41.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.42.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.43.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.44.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.45.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.46.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.47.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.48.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.49.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.5.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.50.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.51.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.52.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.53.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.54.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.55.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.56.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.57.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.58.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.59.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.6.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.60.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.61.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.62.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.63.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.64.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.65.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.66.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.67.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.7.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.8.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.down_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.down_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.gate_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.gate_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.up_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.mlp.experts.9.up_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.k_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.k_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.o_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.o_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.q_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.q_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.v_proj.weight": "model-00005-of-00042.safetensors",
+ "model.layers.5.self_attn.v_proj.weight_scale": "model-00005-of-00042.safetensors",
+ "model.layers.1.mlp.gate.e_score_correction_bias": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.gate.e_score_correction_bias": "model-00003-of-00042.safetensors",
+ "model.layers.1.input_layernorm.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.gate.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.shared_experts.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.shared_experts.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.shared_experts.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.mlp.shared_experts.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.1.post_attention_layernorm.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.input_layernorm.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.0.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.1.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.10.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.100.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.101.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.102.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.103.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.104.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.105.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.106.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.107.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.108.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.109.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.11.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.110.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.111.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.112.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.113.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.114.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.115.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.116.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.117.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.118.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.119.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.12.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.120.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.121.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.122.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.123.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.124.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.125.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.126.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.127.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.13.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.14.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.15.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.16.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.17.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.18.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.19.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.2.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.20.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.21.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.22.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.23.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.24.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.25.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.26.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.27.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.28.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.29.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.3.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.30.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.31.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.32.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.33.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.34.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.35.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.36.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.37.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.38.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.39.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.4.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.40.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.41.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.42.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.43.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.44.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.45.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.46.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.47.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.48.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.49.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.5.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.50.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.51.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.52.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.53.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.54.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.55.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.56.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.57.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.58.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.59.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.6.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.60.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.61.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.62.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.63.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.64.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.65.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.66.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.67.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.68.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.69.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.7.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.70.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.71.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.72.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.73.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.74.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.75.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.76.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.77.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.78.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.79.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.8.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.80.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.81.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.82.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.83.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.84.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.85.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.86.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.87.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.88.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.89.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.9.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.90.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.91.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.92.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.93.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.94.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.95.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.96.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.97.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.98.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.experts.99.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.gate.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.shared_experts.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.shared_experts.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.shared_experts.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.mlp.shared_experts.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.post_attention_layernorm.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.self_attn.k_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.self_attn.o_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.self_attn.o_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.2.self_attn.q_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.2.self_attn.v_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.0.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.1.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.10.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.11.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.12.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.13.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.14.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.15.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.16.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.17.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.2.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.3.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.4.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.5.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.6.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.7.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.8.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.down_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.down_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.gate_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.gate_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.up_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.mlp.experts.9.up_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.k_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.k_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.o_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.o_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.q_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.q_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.v_proj.weight": "model-00003-of-00042.safetensors",
+ "model.layers.3.self_attn.v_proj.weight_scale": "model-00003-of-00042.safetensors",
+ "model.layers.5.mlp.gate.e_score_correction_bias": "model-00006-of-00042.safetensors",
+ "model.layers.5.input_layernorm.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.100.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.101.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.102.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.103.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.104.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.105.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.106.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.107.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.108.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.109.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.110.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.111.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.112.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.113.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.114.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.115.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.116.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.117.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.118.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.119.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.120.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.121.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.122.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.123.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.124.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.125.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.126.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.127.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.68.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.69.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.70.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.71.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.72.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.73.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.74.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.75.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.76.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.77.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.78.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.79.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.80.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.81.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.82.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.83.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.84.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.85.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.86.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.87.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.88.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.89.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.90.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.91.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.92.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.93.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.94.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.95.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.96.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.97.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.98.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.experts.99.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.gate.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.shared_experts.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.shared_experts.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.shared_experts.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.mlp.shared_experts.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.5.post_attention_layernorm.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.0.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.1.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.10.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.11.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.12.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.13.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.14.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.15.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.16.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.17.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.18.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.19.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.2.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.20.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.21.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.22.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.23.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.24.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.25.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.26.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.27.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.28.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.29.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.3.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.30.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.31.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.32.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.33.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.34.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.35.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.36.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.37.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.38.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.39.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.4.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.40.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.41.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.42.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.43.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.44.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.45.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.46.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.47.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.48.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.49.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.5.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.50.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.51.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.52.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.53.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.54.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.55.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.56.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.57.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.58.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.59.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.6.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.60.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.61.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.62.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.63.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.64.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.65.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.66.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.67.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.68.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.69.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.7.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.70.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.71.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.72.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.73.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.74.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.75.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.76.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.77.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.78.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.79.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.8.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.80.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.81.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.82.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.83.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.84.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.85.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.86.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.87.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.88.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.89.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.9.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.90.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.91.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.down_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.down_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.92.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.gate_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.gate_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.up_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.up_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.k_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.k_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.o_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.o_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.q_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.q_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.v_proj.weight": "model-00006-of-00042.safetensors",
+ "model.layers.6.self_attn.v_proj.weight_scale": "model-00006-of-00042.safetensors",
+ "model.layers.7.mlp.gate.e_score_correction_bias": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.gate.e_score_correction_bias": "model-00008-of-00042.safetensors",
+ "model.layers.7.input_layernorm.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.119.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.120.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.121.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.122.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.123.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.124.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.125.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.126.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.experts.127.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.gate.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.shared_experts.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.shared_experts.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.mlp.shared_experts.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.7.post_attention_layernorm.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.input_layernorm.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.0.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.1.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.10.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.100.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.101.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.102.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.103.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.104.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.105.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.106.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.107.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.108.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.109.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.11.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.110.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.111.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.112.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.113.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.114.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.115.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.116.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.117.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.118.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.119.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.12.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.120.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.121.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.122.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.123.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.124.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.125.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.126.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.127.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.13.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.14.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.15.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.16.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.17.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.18.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.19.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.2.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.20.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.21.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.22.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.23.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.24.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.25.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.26.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.27.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.28.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.29.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.3.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.30.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.31.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.32.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.33.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.34.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.35.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.36.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.37.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.38.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.39.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.4.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.40.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.41.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.42.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.43.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.44.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.45.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.46.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.47.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.48.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.49.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.5.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.50.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.51.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.52.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.53.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.54.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.55.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.56.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.57.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.58.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.59.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.6.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.60.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.61.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.62.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.63.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.64.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.65.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.66.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.67.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.68.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.69.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.7.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.70.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.71.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.72.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.73.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.74.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.75.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.76.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.77.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.78.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.79.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.8.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.80.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.81.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.82.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.83.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.84.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.85.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.86.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.87.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.88.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.89.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.9.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.90.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.91.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.92.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.93.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.94.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.95.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.96.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.97.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.98.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.experts.99.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.gate.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.mlp.shared_experts.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.post_attention_layernorm.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.k_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.k_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.o_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.o_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.q_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.q_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.v_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.8.self_attn.v_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.0.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.1.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.2.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.3.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.4.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.5.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.6.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.7.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.down_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.down_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.8.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.gate_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.gate_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.up_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.up_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.k_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.k_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.o_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.o_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.q_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.q_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.v_proj.weight": "model-00008-of-00042.safetensors",
+ "model.layers.9.self_attn.v_proj.weight_scale": "model-00008-of-00042.safetensors",
+ "model.layers.0.mlp.gate.e_score_correction_bias": "model-00002-of-00042.safetensors",
+ "model.layers.0.input_layernorm.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.103.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.104.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.105.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.106.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.107.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.108.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.109.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.110.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.111.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.112.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.113.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.114.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.115.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.116.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.117.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.118.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.119.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.120.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.121.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.122.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.123.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.124.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.125.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.126.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.experts.127.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.gate.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.shared_experts.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.shared_experts.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.shared_experts.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.mlp.shared_experts.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.0.post_attention_layernorm.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.0.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.1.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.10.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.100.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.101.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.102.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.103.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.104.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.105.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.106.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.107.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.108.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.109.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.11.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.110.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.111.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.112.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.113.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.114.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.115.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.116.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.117.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.118.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.119.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.12.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.120.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.121.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.122.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.123.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.124.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.125.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.126.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.127.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.13.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.14.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.15.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.16.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.17.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.18.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.19.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.2.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.20.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.21.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.22.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.23.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.24.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.25.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.26.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.27.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.28.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.29.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.3.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.30.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.31.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.32.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.33.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.34.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.35.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.36.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.37.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.38.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.39.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.4.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.40.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.41.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.42.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.43.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.44.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.45.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.46.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.47.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.48.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.49.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.5.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.50.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.51.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.52.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.53.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.54.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.55.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.56.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.57.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.58.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.59.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.6.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.60.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.61.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.62.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.63.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.64.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.65.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.66.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.67.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.68.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.69.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.7.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.70.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.71.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.72.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.73.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.74.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.75.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.76.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.77.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.78.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.79.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.8.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.80.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.81.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.82.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.83.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.84.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.85.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.86.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.87.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.88.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.89.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.9.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.90.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.91.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.92.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.93.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.94.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.95.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.96.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.97.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.98.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.down_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.down_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.gate_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.gate_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.up_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.mlp.experts.99.up_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.self_attn.k_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.self_attn.o_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.self_attn.o_proj.weight_scale": "model-00002-of-00042.safetensors",
+ "model.layers.1.self_attn.q_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.1.self_attn.v_proj.weight": "model-00002-of-00042.safetensors",
+ "model.layers.3.mlp.gate.e_score_correction_bias": "model-00004-of-00042.safetensors",
+ "model.layers.3.input_layernorm.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.100.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.101.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.102.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.103.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.104.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.105.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.106.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.107.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.108.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.109.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.110.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.111.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.112.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.113.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.114.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.115.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.116.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.117.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.118.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.119.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.120.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.121.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.122.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.123.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.124.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.125.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.126.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.127.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.18.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.19.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.20.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.21.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.22.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.23.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.24.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.25.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.26.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.27.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.28.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.29.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.30.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.31.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.32.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.33.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.34.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.35.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.36.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.37.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.38.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.39.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.40.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.41.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.42.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.43.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.44.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.45.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.46.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.47.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.48.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.49.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.50.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.51.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.52.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.53.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.54.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.55.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.56.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.57.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.58.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.59.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.60.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.61.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.62.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.63.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.64.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.65.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.66.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.67.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.68.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.69.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.70.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.71.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.72.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.73.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.74.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.75.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.76.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.77.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.78.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.79.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.80.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.81.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.82.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.83.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.84.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.85.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.86.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.87.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.88.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.89.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.90.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.91.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.92.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.93.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.94.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.95.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.96.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.97.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.98.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.experts.99.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.gate.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.shared_experts.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.shared_experts.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.shared_experts.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.mlp.shared_experts.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.3.post_attention_layernorm.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.0.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.1.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.10.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.11.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.12.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.13.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.14.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.15.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.16.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.17.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.18.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.19.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.2.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.20.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.21.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.22.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.23.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.24.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.25.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.26.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.27.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.28.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.29.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.3.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.30.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.31.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.32.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.33.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.34.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.35.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.36.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.37.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.38.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.39.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.4.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.40.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.41.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.42.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.43.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.5.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.6.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.7.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.8.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.down_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.down_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.gate_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.gate_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.up_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.mlp.experts.9.up_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.k_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.k_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.o_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.o_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.q_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.q_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.v_proj.weight": "model-00004-of-00042.safetensors",
+ "model.layers.4.self_attn.v_proj.weight_scale": "model-00004-of-00042.safetensors",
+ "model.layers.6.mlp.gate.e_score_correction_bias": "model-00007-of-00042.safetensors",
+ "model.layers.6.input_layernorm.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.100.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.101.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.102.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.103.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.104.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.105.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.106.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.107.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.108.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.109.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.110.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.111.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.112.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.113.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.114.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.115.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.116.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.117.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.118.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.119.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.120.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.121.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.122.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.123.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.124.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.125.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.126.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.127.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.93.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.94.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.95.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.96.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.97.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.98.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.experts.99.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.gate.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.shared_experts.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.shared_experts.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.mlp.shared_experts.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.6.post_attention_layernorm.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.0.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.1.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.10.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.100.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.101.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.102.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.103.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.104.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.105.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.106.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.107.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.108.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.109.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.11.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.110.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.111.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.112.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.113.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.114.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.115.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.116.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.117.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.118.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.12.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.13.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.14.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.15.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.16.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.17.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.18.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.19.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.2.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.20.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.21.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.22.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.23.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.24.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.25.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.26.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.27.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.28.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.29.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.3.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.30.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.31.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.32.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.33.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.34.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.35.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.36.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.37.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.38.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.39.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.4.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.40.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.41.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.42.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.43.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.44.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.45.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.46.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.47.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.48.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.49.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.5.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.50.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.51.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.52.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.53.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.54.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.55.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.56.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.57.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.58.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.59.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.6.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.60.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.61.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.62.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.63.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.64.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.65.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.66.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.67.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.68.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.69.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.7.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.70.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.71.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.72.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.73.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.74.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.75.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.76.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.77.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.78.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.79.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.8.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.80.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.81.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.82.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.83.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.84.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.85.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.86.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.87.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.88.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.89.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.9.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.90.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.91.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.92.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.93.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.94.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.95.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.96.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.97.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.98.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.down_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.down_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.gate_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.gate_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.up_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.mlp.experts.99.up_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.k_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.k_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.o_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.o_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.q_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.q_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.v_proj.weight": "model-00007-of-00042.safetensors",
+ "model.layers.7.self_attn.v_proj.weight_scale": "model-00007-of-00042.safetensors",
+ "model.embed_tokens.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.0.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.1.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.10.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.100.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.101.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.102.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.11.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.12.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.13.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.14.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.15.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.16.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.17.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.18.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.19.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.2.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.20.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.21.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.22.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.23.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.24.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.25.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.26.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.27.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.28.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.29.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.3.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.30.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.31.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.32.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.33.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.34.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.35.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.36.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.37.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.38.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.39.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.4.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.40.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.41.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.42.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.43.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.44.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.45.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.46.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.47.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.48.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.49.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.5.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.50.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.51.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.52.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.53.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.54.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.55.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.56.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.57.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.58.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.59.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.6.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.60.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.61.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.62.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.63.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.64.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.65.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.66.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.67.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.68.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.69.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.7.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.70.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.71.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.72.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.73.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.74.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.75.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.76.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.77.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.78.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.79.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.8.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.80.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.81.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.82.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.83.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.84.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.85.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.86.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.87.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.88.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.89.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.9.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.90.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.91.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.92.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.93.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.94.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.95.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.96.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.97.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.98.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.down_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.down_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.gate_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.gate_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.up_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.mlp.experts.99.up_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.self_attn.o_proj.weight_scale": "model-00001-of-00042.safetensors",
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00042.safetensors",
+ "model.layers.10.mlp.gate.e_score_correction_bias": "model-00010-of-00042.safetensors",
+ "model.layers.10.input_layernorm.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.100.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.101.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.102.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.103.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.104.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.105.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.106.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.107.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.108.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.109.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.110.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.111.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.112.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.113.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.114.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.115.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.116.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.117.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.118.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.119.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.120.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.121.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.122.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.123.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.124.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.125.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.126.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.127.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.35.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.36.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.37.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.38.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.39.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.40.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.41.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.42.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.43.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.44.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.45.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.46.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.47.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.48.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.49.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.50.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.51.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.52.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.53.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.54.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.55.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.56.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.57.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.58.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.59.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.60.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.61.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.62.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.63.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.64.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.65.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.66.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.67.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.68.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.69.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.70.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.71.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.72.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.73.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.74.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.75.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.76.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.77.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.78.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.79.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.80.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.81.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.82.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.83.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.84.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.85.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.86.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.87.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.88.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.89.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.90.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.91.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.92.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.93.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.94.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.95.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.96.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.97.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.98.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.experts.99.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.gate.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.10.mlp.shared_experts.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.10.post_attention_layernorm.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.0.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.1.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.10.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.11.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.12.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.13.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.14.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.15.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.16.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.17.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.18.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.19.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.2.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.20.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.21.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.22.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.23.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.24.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.25.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.26.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.27.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.28.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.29.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.3.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.30.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.31.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.32.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.33.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.34.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.35.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.36.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.37.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.38.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.39.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.4.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.40.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.41.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.42.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.43.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.44.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.45.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.46.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.47.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.48.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.49.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.5.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.50.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.51.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.52.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.53.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.54.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.55.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.56.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.57.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.58.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.6.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.7.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.8.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.down_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.down_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.gate_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.gate_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.up_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.mlp.experts.9.up_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.k_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.k_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.o_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.o_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.q_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.q_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.v_proj.weight": "model-00010-of-00042.safetensors",
+ "model.layers.11.self_attn.v_proj.weight_scale": "model-00010-of-00042.safetensors",
+ "model.layers.12.mlp.gate.e_score_correction_bias": "model-00012-of-00042.safetensors",
+ "model.layers.12.input_layernorm.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.100.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.101.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.102.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.103.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.104.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.105.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.106.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.107.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.108.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.109.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.110.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.111.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.112.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.113.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.114.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.115.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.116.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.117.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.118.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.119.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.120.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.121.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.122.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.123.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.124.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.125.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.126.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.127.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.85.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.86.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.87.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.88.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.89.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.90.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.91.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.92.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.93.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.94.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.95.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.96.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.97.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.98.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.experts.99.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.gate.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.12.mlp.shared_experts.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.12.post_attention_layernorm.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.0.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.1.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.10.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.100.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.101.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.102.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.103.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.104.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.105.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.106.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.107.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.108.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.11.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.12.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.13.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.14.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.15.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.16.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.17.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.18.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.19.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.2.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.20.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.21.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.22.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.23.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.24.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.25.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.26.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.27.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.28.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.29.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.3.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.30.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.31.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.32.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.33.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.34.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.35.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.36.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.37.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.38.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.39.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.4.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.40.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.41.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.42.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.43.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.44.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.45.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.46.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.47.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.48.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.49.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.5.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.50.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.51.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.52.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.53.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.54.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.55.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.56.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.57.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.58.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.59.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.6.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.60.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.61.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.62.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.63.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.64.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.65.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.66.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.67.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.68.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.69.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.7.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.70.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.71.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.72.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.73.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.74.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.75.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.76.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.77.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.78.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.79.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.8.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.80.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.81.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.82.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.83.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.84.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.85.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.86.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.87.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.88.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.89.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.9.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.90.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.91.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.92.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.93.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.94.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.95.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.96.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.97.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.98.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.down_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.down_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.gate_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.gate_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.up_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.mlp.experts.99.up_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.k_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.k_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.o_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.o_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.q_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.q_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.v_proj.weight": "model-00012-of-00042.safetensors",
+ "model.layers.13.self_attn.v_proj.weight_scale": "model-00012-of-00042.safetensors",
+ "model.layers.11.mlp.gate.e_score_correction_bias": "model-00011-of-00042.safetensors",
+ "model.layers.11.input_layernorm.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.100.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.101.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.102.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.103.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.104.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.105.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.106.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.107.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.108.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.109.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.110.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.111.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.112.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.113.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.114.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.115.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.116.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.117.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.118.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.119.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.120.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.121.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.122.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.123.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.124.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.125.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.126.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.127.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.59.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.60.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.61.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.62.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.63.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.64.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.65.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.66.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.67.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.68.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.69.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.70.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.71.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.72.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.73.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.74.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.75.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.76.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.77.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.78.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.79.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.80.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.81.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.82.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.83.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.84.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.85.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.86.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.87.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.88.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.89.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.90.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.91.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.92.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.93.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.94.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.95.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.96.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.97.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.98.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.experts.99.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.gate.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.11.mlp.shared_experts.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.11.post_attention_layernorm.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.0.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.1.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.10.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.11.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.12.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.13.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.14.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.15.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.16.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.17.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.18.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.19.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.2.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.20.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.21.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.22.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.23.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.24.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.25.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.26.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.27.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.28.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.29.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.3.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.30.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.31.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.32.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.33.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.34.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.35.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.36.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.37.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.38.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.39.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.4.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.40.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.41.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.42.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.43.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.44.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.45.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.46.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.47.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.48.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.49.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.5.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.50.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.51.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.52.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.53.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.54.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.55.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.56.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.57.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.58.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.59.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.6.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.60.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.61.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.62.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.63.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.64.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.65.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.66.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.67.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.68.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.69.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.7.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.70.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.71.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.72.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.73.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.74.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.75.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.76.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.77.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.78.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.79.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.8.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.80.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.81.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.82.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.83.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.84.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.down_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.down_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.gate_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.gate_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.up_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.mlp.experts.9.up_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.k_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.k_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.o_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.o_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.q_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.q_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.v_proj.weight": "model-00011-of-00042.safetensors",
+ "model.layers.12.self_attn.v_proj.weight_scale": "model-00011-of-00042.safetensors",
+ "model.layers.9.mlp.gate.e_score_correction_bias": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.0.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.1.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.10.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.11.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.12.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.13.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.14.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.15.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.16.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.17.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.18.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.19.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.2.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.20.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.21.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.22.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.23.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.24.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.25.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.26.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.27.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.28.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.29.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.3.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.30.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.31.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.32.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.33.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.34.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.4.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.5.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.6.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.7.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.8.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.mlp.experts.9.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.k_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.k_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.o_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.o_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.q_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.q_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.v_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.10.self_attn.v_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.input_layernorm.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.10.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.100.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.101.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.102.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.103.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.104.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.105.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.106.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.107.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.108.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.109.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.11.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.110.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.111.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.112.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.113.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.114.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.115.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.116.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.117.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.118.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.119.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.12.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.120.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.121.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.122.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.123.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.124.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.125.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.126.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.127.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.13.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.14.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.15.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.16.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.17.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.18.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.19.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.20.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.21.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.22.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.23.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.24.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.25.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.26.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.27.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.28.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.29.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.30.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.31.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.32.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.33.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.34.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.35.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.36.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.37.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.38.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.39.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.40.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.41.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.42.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.43.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.44.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.45.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.46.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.47.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.48.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.49.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.50.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.51.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.52.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.53.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.54.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.55.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.56.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.57.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.58.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.59.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.60.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.61.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.62.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.63.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.64.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.65.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.66.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.67.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.68.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.69.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.70.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.71.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.72.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.73.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.74.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.75.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.76.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.77.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.78.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.79.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.80.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.81.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.82.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.83.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.84.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.85.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.86.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.87.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.88.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.89.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.9.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.90.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.91.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.92.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.93.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.94.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.95.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.96.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.97.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.98.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.experts.99.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.gate.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.down_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.down_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.gate_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.gate_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.up_proj.weight": "model-00009-of-00042.safetensors",
+ "model.layers.9.mlp.shared_experts.up_proj.weight_scale": "model-00009-of-00042.safetensors",
+ "model.layers.9.post_attention_layernorm.weight": "model-00009-of-00042.safetensors",
+ "model.layers.13.mlp.gate.e_score_correction_bias": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.gate.e_score_correction_bias": "model-00013-of-00042.safetensors",
+ "model.layers.13.input_layernorm.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.109.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.110.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.111.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.112.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.113.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.114.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.115.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.116.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.117.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.118.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.119.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.120.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.121.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.122.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.123.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.124.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.125.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.126.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.experts.127.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.gate.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.13.mlp.shared_experts.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.13.post_attention_layernorm.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.input_layernorm.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.0.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.1.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.10.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.100.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.101.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.102.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.103.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.104.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.105.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.106.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.107.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.108.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.109.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.11.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.110.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.111.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.112.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.113.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.114.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.115.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.116.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.117.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.118.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.119.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.12.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.120.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.121.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.122.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.123.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.124.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.125.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.126.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.127.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.13.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.14.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.15.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.16.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.17.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.18.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.19.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.2.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.20.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.21.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.22.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.23.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.24.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.25.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.26.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.27.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.28.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.29.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.3.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.30.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.31.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.32.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.33.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.34.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.35.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.36.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.37.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.38.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.39.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.4.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.40.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.41.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.42.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.43.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.44.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.45.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.46.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.47.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.48.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.49.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.5.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.50.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.51.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.52.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.53.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.54.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.55.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.56.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.57.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.58.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.59.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.6.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.60.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.61.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.62.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.63.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.64.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.65.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.66.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.67.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.68.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.69.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.7.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.70.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.71.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.72.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.73.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.74.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.75.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.76.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.77.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.78.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.79.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.8.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.80.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.81.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.82.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.83.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.84.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.85.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.86.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.87.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.88.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.89.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.9.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.90.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.91.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.92.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.93.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.94.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.95.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.96.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.97.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.98.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.experts.99.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.gate.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.down_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.down_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.mlp.shared_experts.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.post_attention_layernorm.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.k_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.k_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.o_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.o_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.q_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.q_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.v_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.14.self_attn.v_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.gate_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.gate_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.up_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.up_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.k_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.k_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.o_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.o_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.q_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.q_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.v_proj.weight": "model-00013-of-00042.safetensors",
+ "model.layers.15.self_attn.v_proj.weight_scale": "model-00013-of-00042.safetensors",
+ "model.layers.15.mlp.gate.e_score_correction_bias": "model-00014-of-00042.safetensors",
+ "model.layers.15.input_layernorm.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.0.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.1.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.10.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.100.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.101.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.102.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.103.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.104.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.105.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.106.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.107.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.108.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.109.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.11.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.110.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.111.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.112.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.113.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.114.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.115.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.116.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.117.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.118.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.119.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.12.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.120.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.121.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.122.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.123.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.124.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.125.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.126.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.127.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.13.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.14.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.15.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.16.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.17.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.18.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.19.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.2.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.20.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.21.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.22.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.23.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.24.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.25.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.26.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.27.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.28.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.29.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.3.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.30.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.31.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.32.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.33.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.34.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.35.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.36.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.37.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.38.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.39.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.4.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.40.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.41.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.42.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.43.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.44.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.45.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.46.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.47.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.48.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.49.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.5.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.50.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.51.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.52.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.53.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.54.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.55.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.56.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.57.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.58.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.59.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.6.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.60.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.61.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.62.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.63.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.64.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.65.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.66.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.67.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.68.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.69.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.7.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.70.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.71.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.72.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.73.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.74.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.75.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.76.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.77.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.78.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.79.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.8.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.80.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.81.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.82.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.83.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.84.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.85.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.86.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.87.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.88.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.89.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.9.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.90.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.91.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.92.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.93.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.94.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.95.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.96.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.97.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.98.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.experts.99.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.gate.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.15.mlp.shared_experts.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.15.post_attention_layernorm.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.0.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.1.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.10.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.11.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.12.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.13.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.14.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.15.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.16.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.17.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.18.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.19.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.2.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.20.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.21.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.22.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.23.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.24.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.3.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.4.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.5.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.6.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.7.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.8.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.down_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.down_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.gate_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.gate_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.up_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.mlp.experts.9.up_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.k_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.k_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.o_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.o_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.q_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.q_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.v_proj.weight": "model-00014-of-00042.safetensors",
+ "model.layers.16.self_attn.v_proj.weight_scale": "model-00014-of-00042.safetensors",
+ "model.layers.17.mlp.gate.e_score_correction_bias": "model-00016-of-00042.safetensors",
+ "model.layers.17.input_layernorm.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.100.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.101.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.102.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.103.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.104.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.105.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.106.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.107.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.108.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.109.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.110.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.111.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.112.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.113.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.114.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.115.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.116.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.117.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.118.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.119.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.120.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.121.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.122.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.123.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.124.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.125.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.126.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.127.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.51.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.52.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.53.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.54.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.55.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.56.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.57.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.58.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.59.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.60.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.61.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.62.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.63.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.64.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.65.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.66.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.67.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.68.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.69.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.70.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.71.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.72.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.73.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.74.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.75.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.76.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.77.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.78.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.79.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.80.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.81.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.82.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.83.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.84.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.85.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.86.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.87.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.88.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.89.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.90.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.91.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.92.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.93.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.94.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.95.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.96.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.97.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.98.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.experts.99.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.gate.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.17.mlp.shared_experts.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.17.post_attention_layernorm.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.0.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.1.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.10.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.11.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.12.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.13.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.14.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.15.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.16.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.17.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.18.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.19.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.2.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.20.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.21.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.22.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.23.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.24.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.25.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.26.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.27.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.28.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.29.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.3.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.30.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.31.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.32.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.33.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.34.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.35.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.36.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.37.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.38.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.39.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.4.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.40.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.41.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.42.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.43.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.44.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.45.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.46.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.47.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.48.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.49.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.5.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.50.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.51.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.52.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.53.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.54.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.55.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.56.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.57.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.58.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.59.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.6.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.60.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.61.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.62.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.63.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.64.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.65.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.66.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.67.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.68.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.69.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.7.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.70.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.71.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.72.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.73.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.74.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.8.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.down_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.down_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.gate_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.gate_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.up_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.mlp.experts.9.up_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.k_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.k_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.o_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.o_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.q_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.q_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.v_proj.weight": "model-00016-of-00042.safetensors",
+ "model.layers.18.self_attn.v_proj.weight_scale": "model-00016-of-00042.safetensors",
+ "model.layers.16.mlp.gate.e_score_correction_bias": "model-00015-of-00042.safetensors",
+ "model.layers.16.input_layernorm.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.100.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.101.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.102.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.103.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.104.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.105.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.106.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.107.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.108.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.109.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.110.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.111.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.112.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.113.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.114.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.115.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.116.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.117.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.118.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.119.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.120.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.121.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.122.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.123.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.124.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.125.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.126.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.127.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.25.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.26.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.27.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.28.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.29.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.30.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.31.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.32.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.33.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.34.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.35.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.36.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.37.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.38.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.39.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.40.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.41.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.42.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.43.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.44.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.45.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.46.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.47.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.48.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.49.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.50.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.51.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.52.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.53.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.54.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.55.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.56.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.57.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.58.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.59.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.60.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.61.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.62.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.63.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.64.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.65.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.66.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.67.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.68.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.69.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.70.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.71.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.72.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.73.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.74.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.75.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.76.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.77.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.78.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.79.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.80.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.81.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.82.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.83.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.84.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.85.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.86.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.87.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.88.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.89.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.90.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.91.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.92.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.93.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.94.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.95.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.96.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.97.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.98.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.experts.99.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.gate.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.16.mlp.shared_experts.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.16.post_attention_layernorm.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.0.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.1.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.10.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.11.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.12.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.13.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.14.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.15.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.16.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.17.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.18.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.19.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.2.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.20.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.21.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.22.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.23.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.24.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.25.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.26.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.27.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.28.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.29.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.3.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.30.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.31.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.32.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.33.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.34.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.35.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.36.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.37.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.38.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.39.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.4.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.40.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.41.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.42.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.43.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.44.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.45.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.46.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.47.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.48.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.49.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.5.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.50.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.6.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.7.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.8.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.down_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.down_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.gate_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.gate_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.up_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.mlp.experts.9.up_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.k_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.k_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.o_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.o_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.q_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.q_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.v_proj.weight": "model-00015-of-00042.safetensors",
+ "model.layers.17.self_attn.v_proj.weight_scale": "model-00015-of-00042.safetensors",
+ "model.layers.20.mlp.gate.e_score_correction_bias": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.gate.e_score_correction_bias": "model-00019-of-00042.safetensors",
+ "model.layers.20.input_layernorm.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.126.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.experts.127.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.gate.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.20.mlp.shared_experts.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.20.post_attention_layernorm.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.input_layernorm.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.0.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.1.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.10.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.100.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.101.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.102.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.103.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.104.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.105.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.106.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.107.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.108.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.109.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.11.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.110.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.111.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.112.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.113.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.114.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.115.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.116.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.117.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.118.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.119.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.12.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.120.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.121.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.122.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.123.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.124.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.125.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.126.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.127.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.13.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.14.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.15.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.16.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.17.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.18.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.19.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.2.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.20.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.21.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.22.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.23.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.24.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.25.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.26.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.27.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.28.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.29.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.3.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.30.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.31.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.32.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.33.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.34.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.35.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.36.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.37.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.38.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.39.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.4.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.40.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.41.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.42.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.43.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.44.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.45.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.46.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.47.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.48.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.49.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.5.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.50.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.51.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.52.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.53.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.54.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.55.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.56.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.57.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.58.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.59.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.6.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.60.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.61.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.62.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.63.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.64.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.65.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.66.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.67.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.68.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.69.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.7.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.70.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.71.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.72.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.73.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.74.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.75.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.76.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.77.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.78.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.79.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.8.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.80.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.81.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.82.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.83.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.84.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.85.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.86.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.87.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.88.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.89.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.9.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.90.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.91.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.92.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.93.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.94.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.95.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.96.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.97.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.98.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.experts.99.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.gate.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.mlp.shared_experts.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.post_attention_layernorm.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.k_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.k_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.o_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.o_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.q_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.q_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.v_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.21.self_attn.v_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.0.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.1.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.10.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.11.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.12.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.13.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.14.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.15.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.2.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.3.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.4.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.5.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.6.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.7.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.8.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.down_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.down_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.gate_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.gate_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.up_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.mlp.experts.9.up_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.k_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.k_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.o_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.o_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.q_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.q_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.v_proj.weight": "model-00019-of-00042.safetensors",
+ "model.layers.22.self_attn.v_proj.weight_scale": "model-00019-of-00042.safetensors",
+ "model.layers.19.mlp.gate.e_score_correction_bias": "model-00018-of-00042.safetensors",
+ "model.layers.19.input_layernorm.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.101.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.102.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.103.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.104.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.105.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.106.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.107.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.108.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.109.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.110.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.111.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.112.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.113.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.114.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.115.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.116.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.117.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.118.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.119.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.120.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.121.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.122.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.123.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.124.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.125.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.126.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.experts.127.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.gate.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.19.mlp.shared_experts.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.19.post_attention_layernorm.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.0.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.1.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.10.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.100.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.101.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.102.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.103.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.104.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.105.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.106.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.107.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.108.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.109.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.11.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.110.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.111.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.112.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.113.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.114.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.115.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.116.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.117.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.118.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.119.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.12.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.120.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.121.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.122.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.123.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.124.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.125.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.13.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.14.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.15.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.16.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.17.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.18.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.19.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.2.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.20.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.21.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.22.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.23.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.24.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.25.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.26.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.27.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.28.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.29.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.3.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.30.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.31.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.32.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.33.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.34.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.35.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.36.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.37.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.38.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.39.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.4.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.40.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.41.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.42.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.43.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.44.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.45.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.46.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.47.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.48.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.49.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.5.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.50.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.51.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.52.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.53.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.54.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.55.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.56.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.57.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.58.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.59.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.6.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.60.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.61.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.62.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.63.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.64.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.65.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.66.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.67.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.68.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.69.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.7.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.70.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.71.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.72.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.73.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.74.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.75.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.76.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.77.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.78.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.79.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.8.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.80.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.81.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.82.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.83.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.84.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.85.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.86.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.87.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.88.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.89.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.9.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.90.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.91.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.92.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.93.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.94.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.95.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.96.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.97.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.98.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.down_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.down_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.gate_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.gate_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.up_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.mlp.experts.99.up_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.k_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.k_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.o_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.o_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.q_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.q_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.v_proj.weight": "model-00018-of-00042.safetensors",
+ "model.layers.20.self_attn.v_proj.weight_scale": "model-00018-of-00042.safetensors",
+ "model.layers.18.mlp.gate.e_score_correction_bias": "model-00017-of-00042.safetensors",
+ "model.layers.18.input_layernorm.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.100.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.101.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.102.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.103.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.104.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.105.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.106.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.107.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.108.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.109.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.110.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.111.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.112.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.113.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.114.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.115.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.116.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.117.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.118.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.119.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.120.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.121.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.122.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.123.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.124.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.125.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.126.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.127.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.75.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.76.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.77.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.78.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.79.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.80.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.81.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.82.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.83.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.84.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.85.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.86.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.87.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.88.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.89.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.90.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.91.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.92.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.93.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.94.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.95.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.96.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.97.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.98.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.experts.99.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.gate.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.18.mlp.shared_experts.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.18.post_attention_layernorm.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.0.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.1.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.10.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.100.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.11.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.12.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.13.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.14.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.15.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.16.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.17.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.18.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.19.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.2.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.20.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.21.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.22.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.23.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.24.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.25.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.26.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.27.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.28.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.29.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.3.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.30.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.31.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.32.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.33.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.34.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.35.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.36.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.37.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.38.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.39.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.4.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.40.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.41.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.42.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.43.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.44.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.45.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.46.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.47.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.48.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.49.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.5.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.50.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.51.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.52.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.53.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.54.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.55.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.56.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.57.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.58.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.59.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.6.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.60.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.61.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.62.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.63.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.64.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.65.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.66.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.67.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.68.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.69.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.7.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.70.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.71.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.72.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.73.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.74.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.75.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.76.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.77.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.78.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.79.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.8.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.80.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.81.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.82.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.83.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.84.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.85.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.86.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.87.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.88.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.89.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.9.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.90.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.91.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.92.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.93.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.94.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.95.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.96.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.97.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.98.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.down_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.down_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.gate_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.gate_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.up_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.mlp.experts.99.up_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.k_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.k_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.o_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.o_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.q_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.q_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.v_proj.weight": "model-00017-of-00042.safetensors",
+ "model.layers.19.self_attn.v_proj.weight_scale": "model-00017-of-00042.safetensors",
+ "model.layers.22.mlp.gate.e_score_correction_bias": "model-00020-of-00042.safetensors",
+ "model.layers.22.input_layernorm.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.100.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.101.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.102.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.103.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.104.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.105.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.106.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.107.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.108.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.109.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.110.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.111.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.112.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.113.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.114.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.115.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.116.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.117.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.118.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.119.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.120.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.121.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.122.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.123.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.124.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.125.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.126.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.127.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.16.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.17.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.18.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.19.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.20.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.21.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.22.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.23.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.24.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.25.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.26.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.27.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.28.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.29.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.30.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.31.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.32.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.33.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.34.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.35.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.36.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.37.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.38.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.39.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.40.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.41.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.42.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.43.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.44.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.45.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.46.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.47.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.48.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.49.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.50.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.51.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.52.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.53.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.54.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.55.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.56.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.57.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.58.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.59.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.60.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.61.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.62.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.63.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.64.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.65.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.66.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.67.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.68.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.69.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.70.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.71.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.72.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.73.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.74.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.75.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.76.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.77.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.78.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.79.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.80.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.81.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.82.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.83.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.84.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.85.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.86.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.87.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.88.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.89.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.90.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.91.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.92.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.93.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.94.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.95.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.96.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.97.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.98.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.experts.99.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.gate.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.22.mlp.shared_experts.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.22.post_attention_layernorm.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.0.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.1.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.10.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.11.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.12.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.13.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.14.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.15.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.16.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.17.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.18.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.19.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.2.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.20.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.21.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.22.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.23.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.24.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.25.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.26.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.27.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.28.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.29.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.3.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.30.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.31.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.32.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.33.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.34.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.35.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.36.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.37.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.38.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.39.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.4.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.40.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.5.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.6.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.7.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.8.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.down_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.down_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.gate_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.gate_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.up_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.experts.9.up_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.k_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.k_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.o_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.o_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.q_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.q_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.v_proj.weight": "model-00020-of-00042.safetensors",
+ "model.layers.23.self_attn.v_proj.weight_scale": "model-00020-of-00042.safetensors",
+ "model.layers.23.mlp.gate.e_score_correction_bias": "model-00021-of-00042.safetensors",
+ "model.layers.23.input_layernorm.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.100.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.101.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.102.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.103.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.104.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.105.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.106.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.107.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.108.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.109.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.110.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.111.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.112.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.113.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.114.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.115.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.116.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.117.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.118.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.119.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.120.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.121.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.122.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.123.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.124.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.125.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.126.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.127.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.41.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.42.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.43.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.44.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.45.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.46.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.47.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.48.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.49.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.50.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.51.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.52.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.53.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.54.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.55.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.56.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.57.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.58.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.59.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.60.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.61.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.62.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.63.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.64.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.65.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.66.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.67.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.68.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.69.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.70.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.71.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.72.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.73.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.74.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.75.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.76.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.77.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.78.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.79.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.80.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.81.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.82.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.83.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.84.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.85.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.86.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.87.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.88.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.89.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.90.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.91.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.92.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.93.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.94.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.95.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.96.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.97.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.98.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.experts.99.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.gate.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.23.mlp.shared_experts.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.23.post_attention_layernorm.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.0.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.1.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.10.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.11.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.12.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.13.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.14.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.15.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.16.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.17.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.18.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.19.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.2.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.20.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.21.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.22.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.23.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.24.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.25.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.26.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.27.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.28.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.29.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.3.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.30.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.31.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.32.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.33.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.34.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.35.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.36.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.37.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.38.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.39.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.4.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.40.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.41.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.42.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.43.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.44.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.45.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.46.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.47.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.48.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.49.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.5.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.50.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.51.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.52.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.53.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.54.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.55.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.56.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.57.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.58.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.59.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.6.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.60.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.61.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.62.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.63.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.64.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.65.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.7.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.8.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.down_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.down_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.gate_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.gate_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.up_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.mlp.experts.9.up_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.k_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.k_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.o_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.o_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.q_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.q_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.v_proj.weight": "model-00021-of-00042.safetensors",
+ "model.layers.24.self_attn.v_proj.weight_scale": "model-00021-of-00042.safetensors",
+ "model.layers.25.mlp.gate.e_score_correction_bias": "model-00023-of-00042.safetensors",
+ "model.layers.25.input_layernorm.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.100.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.101.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.102.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.103.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.104.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.105.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.106.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.107.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.108.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.109.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.110.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.111.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.112.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.113.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.114.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.115.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.116.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.117.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.118.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.119.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.120.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.121.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.122.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.123.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.124.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.125.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.126.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.127.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.92.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.93.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.94.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.95.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.96.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.97.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.98.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.experts.99.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.gate.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.25.mlp.shared_experts.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.25.post_attention_layernorm.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.0.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.1.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.10.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.100.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.101.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.102.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.103.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.104.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.105.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.106.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.107.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.108.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.109.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.11.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.110.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.111.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.112.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.113.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.114.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.115.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.12.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.13.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.14.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.15.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.16.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.17.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.18.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.19.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.2.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.20.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.21.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.22.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.23.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.24.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.25.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.26.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.27.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.28.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.29.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.3.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.30.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.31.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.32.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.33.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.34.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.35.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.36.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.37.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.38.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.39.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.4.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.40.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.41.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.42.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.43.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.44.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.45.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.46.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.47.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.48.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.49.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.5.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.50.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.51.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.52.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.53.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.54.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.55.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.56.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.57.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.58.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.59.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.6.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.60.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.61.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.62.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.63.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.64.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.65.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.66.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.67.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.68.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.69.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.7.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.70.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.71.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.72.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.73.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.74.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.75.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.76.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.77.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.78.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.79.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.8.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.80.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.81.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.82.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.83.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.84.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.85.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.86.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.87.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.88.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.89.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.9.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.90.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.91.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.92.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.93.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.94.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.95.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.96.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.97.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.98.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.down_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.down_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.gate_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.gate_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.up_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.experts.99.up_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.k_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.k_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.o_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.o_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.q_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.q_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.v_proj.weight": "model-00023-of-00042.safetensors",
+ "model.layers.26.self_attn.v_proj.weight_scale": "model-00023-of-00042.safetensors",
+ "model.layers.26.mlp.gate.e_score_correction_bias": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.gate.e_score_correction_bias": "model-00024-of-00042.safetensors",
+ "model.layers.26.input_layernorm.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.116.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.117.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.118.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.119.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.120.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.121.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.122.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.123.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.124.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.125.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.126.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.experts.127.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.gate.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.26.mlp.shared_experts.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.26.post_attention_layernorm.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.input_layernorm.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.0.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.1.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.10.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.100.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.101.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.102.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.103.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.104.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.105.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.106.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.107.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.108.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.109.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.11.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.110.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.111.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.112.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.113.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.114.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.115.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.116.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.117.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.118.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.119.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.12.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.120.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.121.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.122.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.123.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.124.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.125.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.126.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.127.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.13.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.14.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.15.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.16.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.17.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.18.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.19.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.2.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.20.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.21.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.22.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.23.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.24.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.25.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.26.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.27.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.28.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.29.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.3.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.30.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.31.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.32.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.33.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.34.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.35.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.36.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.37.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.38.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.39.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.4.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.40.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.41.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.42.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.43.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.44.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.45.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.46.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.47.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.48.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.49.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.5.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.50.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.51.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.52.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.53.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.54.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.55.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.56.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.57.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.58.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.59.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.6.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.60.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.61.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.62.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.63.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.64.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.65.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.66.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.67.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.68.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.69.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.7.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.70.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.71.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.72.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.73.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.74.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.75.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.76.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.77.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.78.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.79.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.8.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.80.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.81.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.82.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.83.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.84.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.85.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.86.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.87.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.88.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.89.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.9.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.90.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.91.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.92.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.93.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.94.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.95.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.96.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.97.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.98.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.experts.99.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.gate.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.mlp.shared_experts.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.post_attention_layernorm.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.k_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.k_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.o_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.o_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.q_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.q_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.v_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.27.self_attn.v_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.0.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.1.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.2.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.3.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.4.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.5.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.down_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.down_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.6.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.gate_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.gate_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.up_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.up_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.k_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.k_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.o_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.o_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.q_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.q_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.v_proj.weight": "model-00024-of-00042.safetensors",
+ "model.layers.28.self_attn.v_proj.weight_scale": "model-00024-of-00042.safetensors",
+ "model.layers.24.mlp.gate.e_score_correction_bias": "model-00022-of-00042.safetensors",
+ "model.layers.24.input_layernorm.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.100.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.101.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.102.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.103.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.104.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.105.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.106.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.107.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.108.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.109.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.110.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.111.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.112.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.113.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.114.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.115.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.116.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.117.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.118.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.119.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.120.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.121.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.122.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.123.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.124.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.125.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.126.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.127.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.66.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.67.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.68.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.69.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.70.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.71.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.72.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.73.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.74.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.75.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.76.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.77.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.78.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.79.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.80.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.81.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.82.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.83.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.84.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.85.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.86.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.87.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.88.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.89.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.90.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.91.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.92.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.93.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.94.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.95.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.96.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.97.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.98.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.experts.99.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.gate.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.24.mlp.shared_experts.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.24.post_attention_layernorm.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.0.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.1.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.10.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.11.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.12.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.13.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.14.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.15.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.16.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.17.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.18.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.19.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.2.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.20.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.21.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.22.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.23.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.24.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.25.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.26.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.27.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.28.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.29.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.3.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.30.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.31.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.32.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.33.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.34.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.35.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.36.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.37.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.38.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.39.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.4.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.40.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.41.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.42.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.43.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.44.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.45.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.46.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.47.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.48.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.49.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.5.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.50.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.51.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.52.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.53.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.54.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.55.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.56.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.57.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.58.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.59.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.6.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.60.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.61.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.62.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.63.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.64.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.65.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.66.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.67.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.68.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.69.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.7.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.70.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.71.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.72.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.73.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.74.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.75.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.76.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.77.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.78.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.79.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.8.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.80.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.81.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.82.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.83.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.84.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.85.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.86.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.87.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.88.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.89.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.9.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.down_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.down_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.90.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.gate_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.gate_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.up_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.mlp.experts.91.up_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.k_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.k_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.o_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.o_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.q_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.q_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.v_proj.weight": "model-00022-of-00042.safetensors",
+ "model.layers.25.self_attn.v_proj.weight_scale": "model-00022-of-00042.safetensors",
+ "model.layers.28.mlp.gate.e_score_correction_bias": "model-00025-of-00042.safetensors",
+ "model.layers.28.input_layernorm.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.10.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.100.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.101.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.102.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.103.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.104.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.105.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.106.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.107.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.108.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.109.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.11.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.110.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.111.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.112.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.113.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.114.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.115.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.116.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.117.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.118.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.119.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.12.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.120.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.121.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.122.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.123.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.124.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.125.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.126.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.127.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.13.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.14.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.15.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.16.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.17.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.18.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.19.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.20.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.21.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.22.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.23.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.24.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.25.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.26.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.27.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.28.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.29.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.30.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.31.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.32.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.33.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.34.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.35.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.36.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.37.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.38.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.39.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.40.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.41.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.42.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.43.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.44.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.45.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.46.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.47.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.48.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.49.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.50.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.51.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.52.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.53.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.54.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.55.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.56.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.57.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.58.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.59.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.60.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.61.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.62.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.63.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.64.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.65.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.66.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.67.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.68.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.69.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.7.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.70.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.71.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.72.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.73.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.74.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.75.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.76.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.77.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.78.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.79.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.8.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.80.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.81.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.82.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.83.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.84.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.85.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.86.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.87.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.88.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.89.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.9.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.90.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.91.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.92.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.93.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.94.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.95.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.96.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.97.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.98.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.experts.99.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.gate.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.28.mlp.shared_experts.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.28.post_attention_layernorm.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.0.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.1.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.10.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.11.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.12.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.13.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.14.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.15.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.16.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.17.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.18.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.19.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.2.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.20.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.21.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.22.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.23.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.24.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.25.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.26.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.27.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.28.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.29.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.3.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.30.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.31.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.4.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.5.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.6.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.7.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.8.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.down_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.down_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.gate_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.gate_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.up_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.experts.9.up_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.k_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.k_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.o_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.o_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.q_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.q_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.v_proj.weight": "model-00025-of-00042.safetensors",
+ "model.layers.29.self_attn.v_proj.weight_scale": "model-00025-of-00042.safetensors",
+ "model.layers.29.mlp.gate.e_score_correction_bias": "model-00026-of-00042.safetensors",
+ "model.layers.29.input_layernorm.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.100.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.101.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.102.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.103.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.104.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.105.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.106.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.107.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.108.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.109.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.110.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.111.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.112.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.113.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.114.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.115.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.116.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.117.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.118.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.119.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.120.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.121.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.122.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.123.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.124.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.125.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.126.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.127.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.32.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.33.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.34.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.35.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.36.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.37.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.38.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.39.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.40.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.41.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.42.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.43.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.44.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.45.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.46.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.47.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.48.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.49.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.50.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.51.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.52.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.53.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.54.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.55.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.56.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.57.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.58.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.59.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.60.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.61.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.62.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.63.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.64.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.65.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.66.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.67.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.68.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.69.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.70.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.71.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.72.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.73.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.74.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.75.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.76.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.77.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.78.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.79.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.80.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.81.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.82.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.83.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.84.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.85.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.86.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.87.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.88.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.89.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.90.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.91.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.92.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.93.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.94.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.95.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.96.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.97.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.98.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.experts.99.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.gate.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.29.mlp.shared_experts.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.29.post_attention_layernorm.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.0.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.1.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.10.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.11.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.12.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.13.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.14.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.15.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.16.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.17.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.18.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.19.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.2.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.20.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.21.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.22.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.23.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.24.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.25.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.26.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.27.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.28.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.29.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.3.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.30.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.31.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.32.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.33.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.34.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.35.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.36.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.37.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.38.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.39.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.4.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.40.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.41.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.42.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.43.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.44.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.45.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.46.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.47.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.48.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.49.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.5.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.50.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.51.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.52.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.53.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.54.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.55.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.56.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.6.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.7.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.8.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.down_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.down_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.gate_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.gate_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.up_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.experts.9.up_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.k_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.k_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.o_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.o_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.q_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.q_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.v_proj.weight": "model-00026-of-00042.safetensors",
+ "model.layers.30.self_attn.v_proj.weight_scale": "model-00026-of-00042.safetensors",
+ "model.layers.30.mlp.gate.e_score_correction_bias": "model-00027-of-00042.safetensors",
+ "model.layers.30.input_layernorm.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.100.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.101.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.102.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.103.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.104.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.105.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.106.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.107.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.108.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.109.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.110.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.111.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.112.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.113.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.114.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.115.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.116.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.117.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.118.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.119.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.120.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.121.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.122.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.123.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.124.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.125.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.126.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.127.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.57.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.58.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.59.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.60.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.61.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.62.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.63.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.64.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.65.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.66.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.67.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.68.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.69.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.70.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.71.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.72.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.73.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.74.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.75.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.76.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.77.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.78.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.79.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.80.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.81.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.82.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.83.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.84.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.85.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.86.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.87.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.88.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.89.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.90.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.91.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.92.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.93.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.94.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.95.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.96.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.97.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.98.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.experts.99.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.gate.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.30.mlp.shared_experts.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.30.post_attention_layernorm.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.0.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.1.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.10.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.11.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.12.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.13.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.14.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.15.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.16.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.17.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.18.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.19.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.2.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.20.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.21.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.22.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.23.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.24.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.25.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.26.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.27.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.28.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.29.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.3.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.30.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.31.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.32.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.33.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.34.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.35.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.36.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.37.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.38.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.39.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.4.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.40.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.41.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.42.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.43.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.44.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.45.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.46.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.47.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.48.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.49.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.5.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.50.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.51.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.52.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.53.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.54.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.55.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.56.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.57.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.58.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.59.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.6.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.60.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.61.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.62.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.63.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.64.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.65.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.66.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.67.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.68.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.69.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.7.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.70.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.71.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.72.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.73.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.74.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.75.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.76.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.77.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.78.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.79.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.8.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.80.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.81.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.down_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.down_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.gate_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.gate_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.up_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.experts.9.up_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.k_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.k_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.o_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.o_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.q_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.q_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.v_proj.weight": "model-00027-of-00042.safetensors",
+ "model.layers.31.self_attn.v_proj.weight_scale": "model-00027-of-00042.safetensors",
+ "model.layers.31.mlp.gate.e_score_correction_bias": "model-00028-of-00042.safetensors",
+ "model.layers.31.input_layernorm.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.100.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.101.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.102.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.103.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.104.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.105.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.106.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.107.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.108.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.109.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.110.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.111.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.112.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.113.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.114.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.115.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.116.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.117.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.118.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.119.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.120.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.121.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.122.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.123.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.124.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.125.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.126.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.127.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.82.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.83.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.84.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.85.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.86.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.87.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.88.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.89.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.90.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.91.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.92.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.93.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.94.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.95.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.96.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.97.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.98.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.experts.99.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.gate.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.31.mlp.shared_experts.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.31.post_attention_layernorm.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.0.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.1.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.10.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.100.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.101.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.102.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.103.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.104.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.105.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.106.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.11.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.12.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.13.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.14.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.15.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.16.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.17.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.18.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.19.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.2.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.20.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.21.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.22.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.23.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.24.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.25.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.26.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.27.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.28.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.29.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.3.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.30.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.31.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.32.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.33.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.34.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.35.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.36.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.37.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.38.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.39.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.4.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.40.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.41.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.42.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.43.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.44.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.45.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.46.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.47.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.48.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.49.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.5.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.50.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.51.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.52.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.53.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.54.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.55.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.56.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.57.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.58.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.59.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.6.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.60.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.61.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.62.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.63.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.64.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.65.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.66.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.67.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.68.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.69.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.7.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.70.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.71.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.72.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.73.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.74.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.75.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.76.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.77.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.78.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.79.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.8.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.80.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.81.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.82.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.83.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.84.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.85.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.86.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.87.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.88.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.89.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.9.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.90.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.91.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.92.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.93.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.94.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.95.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.96.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.97.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.98.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.down_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.down_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.gate_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.gate_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.up_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.mlp.experts.99.up_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.k_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.k_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.o_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.o_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.q_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.q_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.v_proj.weight": "model-00028-of-00042.safetensors",
+ "model.layers.32.self_attn.v_proj.weight_scale": "model-00028-of-00042.safetensors",
+ "model.layers.36.mlp.gate.e_score_correction_bias": "model-00032-of-00042.safetensors",
+ "model.layers.36.input_layernorm.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.100.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.101.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.102.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.103.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.104.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.105.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.106.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.107.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.108.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.109.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.110.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.111.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.112.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.113.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.114.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.115.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.116.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.117.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.118.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.119.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.120.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.121.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.122.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.123.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.124.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.125.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.126.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.127.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.48.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.49.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.50.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.51.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.52.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.53.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.54.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.55.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.56.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.57.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.58.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.59.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.60.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.61.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.62.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.63.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.64.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.65.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.66.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.67.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.68.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.69.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.70.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.71.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.72.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.73.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.74.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.75.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.76.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.77.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.78.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.79.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.80.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.81.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.82.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.83.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.84.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.85.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.86.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.87.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.88.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.89.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.90.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.91.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.92.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.93.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.94.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.95.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.96.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.97.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.98.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.experts.99.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.gate.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.36.mlp.shared_experts.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.36.post_attention_layernorm.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.0.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.1.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.10.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.11.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.12.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.13.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.14.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.15.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.16.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.17.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.18.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.19.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.2.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.20.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.21.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.22.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.23.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.24.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.25.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.26.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.27.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.28.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.29.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.3.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.30.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.31.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.32.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.33.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.34.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.35.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.36.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.37.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.38.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.39.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.4.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.40.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.41.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.42.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.43.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.44.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.45.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.46.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.47.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.48.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.49.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.5.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.50.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.51.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.52.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.53.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.54.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.55.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.56.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.57.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.58.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.59.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.6.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.60.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.61.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.62.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.63.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.64.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.65.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.66.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.67.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.68.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.69.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.7.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.70.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.71.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.8.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.down_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.down_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.gate_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.gate_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.up_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.mlp.experts.9.up_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.k_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.k_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.o_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.o_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.q_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.q_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.v_proj.weight": "model-00032-of-00042.safetensors",
+ "model.layers.37.self_attn.v_proj.weight_scale": "model-00032-of-00042.safetensors",
+ "model.layers.32.mlp.gate.e_score_correction_bias": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.gate.e_score_correction_bias": "model-00029-of-00042.safetensors",
+ "model.layers.32.input_layernorm.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.107.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.108.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.109.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.110.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.111.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.112.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.113.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.114.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.115.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.116.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.117.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.118.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.119.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.120.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.121.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.122.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.123.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.124.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.125.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.126.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.experts.127.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.gate.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.32.mlp.shared_experts.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.32.post_attention_layernorm.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.input_layernorm.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.0.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.1.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.10.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.100.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.101.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.102.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.103.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.104.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.105.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.106.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.107.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.108.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.109.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.11.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.110.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.111.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.112.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.113.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.114.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.115.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.116.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.117.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.118.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.119.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.12.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.120.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.121.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.122.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.123.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.124.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.125.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.126.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.127.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.13.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.14.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.15.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.16.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.17.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.18.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.19.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.2.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.20.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.21.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.22.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.23.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.24.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.25.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.26.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.27.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.28.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.29.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.3.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.30.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.31.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.32.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.33.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.34.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.35.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.36.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.37.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.38.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.39.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.4.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.40.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.41.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.42.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.43.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.44.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.45.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.46.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.47.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.48.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.49.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.5.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.50.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.51.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.52.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.53.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.54.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.55.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.56.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.57.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.58.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.59.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.6.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.60.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.61.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.62.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.63.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.64.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.65.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.66.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.67.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.68.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.69.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.7.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.70.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.71.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.72.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.73.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.74.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.75.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.76.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.77.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.78.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.79.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.8.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.80.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.81.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.82.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.83.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.84.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.85.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.86.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.87.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.88.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.89.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.9.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.90.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.91.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.92.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.93.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.94.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.95.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.96.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.97.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.98.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.experts.99.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.gate.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.down_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.down_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.gate_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.gate_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.up_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.mlp.shared_experts.up_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.post_attention_layernorm.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.k_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.k_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.o_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.o_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.q_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.q_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.v_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.33.self_attn.v_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.k_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.k_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.q_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.q_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.v_proj.weight": "model-00029-of-00042.safetensors",
+ "model.layers.34.self_attn.v_proj.weight_scale": "model-00029-of-00042.safetensors",
+ "model.layers.34.mlp.gate.e_score_correction_bias": "model-00030-of-00042.safetensors",
+ "model.layers.34.input_layernorm.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.0.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.1.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.10.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.100.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.101.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.102.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.103.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.104.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.105.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.106.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.107.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.108.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.109.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.11.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.110.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.111.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.112.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.113.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.114.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.115.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.116.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.117.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.118.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.119.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.12.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.120.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.121.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.122.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.123.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.124.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.125.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.126.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.127.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.13.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.14.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.15.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.16.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.17.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.18.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.19.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.2.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.20.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.21.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.22.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.23.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.24.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.25.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.26.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.27.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.28.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.29.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.3.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.30.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.31.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.32.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.33.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.34.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.35.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.36.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.37.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.38.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.39.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.4.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.40.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.41.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.42.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.43.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.44.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.45.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.46.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.47.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.48.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.49.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.5.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.50.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.51.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.52.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.53.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.54.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.55.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.56.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.57.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.58.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.59.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.6.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.60.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.61.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.62.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.63.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.64.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.65.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.66.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.67.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.68.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.69.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.7.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.70.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.71.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.72.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.73.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.74.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.75.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.76.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.77.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.78.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.79.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.8.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.80.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.81.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.82.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.83.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.84.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.85.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.86.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.87.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.88.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.89.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.9.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.90.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.91.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.92.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.93.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.94.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.95.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.96.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.97.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.98.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.experts.99.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.gate.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.mlp.shared_experts.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.34.post_attention_layernorm.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.self_attn.o_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.34.self_attn.o_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.0.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.1.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.10.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.11.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.12.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.13.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.14.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.15.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.16.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.17.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.18.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.19.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.2.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.20.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.21.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.3.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.4.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.5.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.6.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.7.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.8.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.down_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.down_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.gate_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.gate_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.up_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.experts.9.up_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.k_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.k_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.o_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.o_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.q_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.q_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.v_proj.weight": "model-00030-of-00042.safetensors",
+ "model.layers.35.self_attn.v_proj.weight_scale": "model-00030-of-00042.safetensors",
+ "model.layers.35.mlp.gate.e_score_correction_bias": "model-00031-of-00042.safetensors",
+ "model.layers.35.input_layernorm.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.100.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.101.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.102.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.103.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.104.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.105.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.106.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.107.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.108.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.109.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.110.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.111.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.112.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.113.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.114.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.115.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.116.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.117.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.118.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.119.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.120.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.121.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.122.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.123.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.124.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.125.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.126.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.127.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.22.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.23.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.24.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.25.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.26.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.27.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.28.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.29.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.30.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.31.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.32.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.33.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.34.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.35.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.36.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.37.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.38.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.39.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.40.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.41.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.42.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.43.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.44.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.45.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.46.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.47.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.48.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.49.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.50.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.51.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.52.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.53.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.54.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.55.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.56.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.57.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.58.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.59.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.60.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.61.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.62.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.63.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.64.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.65.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.66.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.67.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.68.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.69.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.70.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.71.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.72.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.73.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.74.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.75.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.76.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.77.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.78.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.79.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.80.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.81.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.82.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.83.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.84.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.85.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.86.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.87.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.88.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.89.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.90.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.91.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.92.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.93.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.94.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.95.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.96.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.97.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.98.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.experts.99.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.gate.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.35.mlp.shared_experts.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.35.post_attention_layernorm.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.0.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.1.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.10.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.11.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.12.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.13.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.14.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.15.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.16.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.17.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.18.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.19.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.2.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.20.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.21.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.22.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.23.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.24.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.25.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.26.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.27.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.28.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.29.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.3.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.30.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.31.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.32.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.33.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.34.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.35.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.36.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.37.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.38.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.39.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.4.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.40.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.41.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.42.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.43.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.44.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.45.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.46.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.47.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.5.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.6.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.7.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.8.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.down_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.down_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.gate_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.gate_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.up_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.mlp.experts.9.up_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.k_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.k_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.o_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.o_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.q_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.q_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.v_proj.weight": "model-00031-of-00042.safetensors",
+ "model.layers.36.self_attn.v_proj.weight_scale": "model-00031-of-00042.safetensors",
+ "model.layers.38.mlp.gate.e_score_correction_bias": "model-00034-of-00042.safetensors",
+ "model.layers.38.input_layernorm.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.100.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.101.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.102.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.103.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.104.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.105.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.106.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.107.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.108.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.109.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.110.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.111.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.112.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.113.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.114.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.115.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.116.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.117.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.118.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.119.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.120.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.121.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.122.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.123.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.124.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.125.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.126.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.127.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.98.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.experts.99.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.gate.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.38.mlp.shared_experts.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.38.post_attention_layernorm.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.0.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.1.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.10.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.100.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.101.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.102.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.103.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.104.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.105.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.106.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.107.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.108.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.109.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.11.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.110.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.111.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.112.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.113.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.114.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.115.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.116.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.117.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.118.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.119.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.12.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.120.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.121.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.13.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.14.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.15.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.16.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.17.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.18.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.19.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.2.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.20.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.21.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.22.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.23.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.24.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.25.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.26.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.27.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.28.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.29.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.3.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.30.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.31.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.32.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.33.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.34.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.35.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.36.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.37.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.38.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.39.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.4.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.40.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.41.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.42.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.43.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.44.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.45.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.46.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.47.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.48.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.49.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.5.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.50.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.51.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.52.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.53.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.54.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.55.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.56.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.57.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.58.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.59.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.6.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.60.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.61.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.62.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.63.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.64.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.65.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.66.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.67.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.68.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.69.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.7.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.70.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.71.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.72.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.73.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.74.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.75.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.76.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.77.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.78.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.79.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.8.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.80.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.81.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.82.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.83.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.84.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.85.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.86.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.87.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.88.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.89.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.9.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.90.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.91.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.92.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.93.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.94.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.95.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.96.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.97.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.98.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.down_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.down_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.gate_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.gate_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.up_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.mlp.experts.99.up_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.k_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.k_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.o_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.o_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.q_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.q_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.v_proj.weight": "model-00034-of-00042.safetensors",
+ "model.layers.39.self_attn.v_proj.weight_scale": "model-00034-of-00042.safetensors",
+ "model.layers.42.mlp.gate.e_score_correction_bias": "model-00037-of-00042.safetensors",
+ "model.layers.42.input_layernorm.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.100.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.101.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.102.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.103.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.104.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.105.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.106.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.107.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.108.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.109.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.110.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.111.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.112.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.113.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.114.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.115.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.116.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.117.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.118.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.119.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.120.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.121.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.122.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.123.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.124.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.125.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.126.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.127.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.39.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.40.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.41.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.42.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.43.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.44.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.45.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.46.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.47.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.48.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.49.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.50.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.51.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.52.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.53.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.54.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.55.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.56.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.57.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.58.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.59.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.60.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.61.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.62.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.63.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.64.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.65.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.66.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.67.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.68.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.69.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.70.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.71.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.72.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.73.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.74.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.75.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.76.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.77.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.78.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.79.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.80.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.81.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.82.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.83.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.84.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.85.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.86.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.87.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.88.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.89.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.90.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.91.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.92.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.93.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.94.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.95.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.96.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.97.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.98.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.experts.99.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.gate.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.42.mlp.shared_experts.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.42.post_attention_layernorm.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.0.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.1.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.10.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.11.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.12.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.13.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.14.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.15.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.16.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.17.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.18.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.19.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.2.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.20.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.21.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.22.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.23.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.24.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.25.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.26.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.27.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.28.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.29.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.3.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.30.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.31.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.32.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.33.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.34.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.35.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.36.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.37.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.38.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.39.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.4.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.40.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.41.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.42.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.43.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.44.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.45.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.46.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.47.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.48.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.49.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.5.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.50.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.51.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.52.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.53.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.54.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.55.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.56.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.57.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.58.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.59.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.6.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.60.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.61.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.62.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.7.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.8.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.down_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.down_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.gate_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.gate_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.up_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.mlp.experts.9.up_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.k_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.k_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.o_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.o_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.q_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.q_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.v_proj.weight": "model-00037-of-00042.safetensors",
+ "model.layers.43.self_attn.v_proj.weight_scale": "model-00037-of-00042.safetensors",
+ "model.layers.39.mlp.gate.e_score_correction_bias": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.gate.e_score_correction_bias": "model-00035-of-00042.safetensors",
+ "model.layers.39.input_layernorm.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.122.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.123.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.124.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.125.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.126.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.experts.127.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.gate.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.39.mlp.shared_experts.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.39.post_attention_layernorm.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.input_layernorm.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.0.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.1.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.10.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.100.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.101.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.102.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.103.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.104.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.105.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.106.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.107.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.108.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.109.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.11.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.110.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.111.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.112.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.113.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.114.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.115.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.116.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.117.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.118.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.119.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.12.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.120.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.121.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.122.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.123.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.124.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.125.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.126.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.127.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.13.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.14.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.15.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.16.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.17.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.18.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.19.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.2.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.20.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.21.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.22.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.23.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.24.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.25.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.26.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.27.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.28.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.29.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.3.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.30.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.31.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.32.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.33.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.34.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.35.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.36.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.37.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.38.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.39.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.4.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.40.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.41.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.42.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.43.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.44.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.45.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.46.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.47.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.48.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.49.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.5.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.50.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.51.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.52.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.53.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.54.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.55.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.56.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.57.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.58.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.59.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.6.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.60.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.61.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.62.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.63.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.64.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.65.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.66.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.67.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.68.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.69.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.7.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.70.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.71.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.72.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.73.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.74.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.75.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.76.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.77.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.78.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.79.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.8.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.80.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.81.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.82.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.83.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.84.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.85.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.86.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.87.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.88.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.89.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.9.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.90.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.91.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.92.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.93.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.94.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.95.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.96.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.97.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.98.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.experts.99.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.gate.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.mlp.shared_experts.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.post_attention_layernorm.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.k_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.k_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.o_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.o_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.q_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.q_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.v_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.40.self_attn.v_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.0.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.1.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.10.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.11.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.12.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.2.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.3.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.4.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.5.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.6.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.7.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.8.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.down_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.down_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.gate_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.gate_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.up_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.mlp.experts.9.up_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.k_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.k_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.o_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.o_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.q_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.q_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.v_proj.weight": "model-00035-of-00042.safetensors",
+ "model.layers.41.self_attn.v_proj.weight_scale": "model-00035-of-00042.safetensors",
+ "model.layers.37.mlp.gate.e_score_correction_bias": "model-00033-of-00042.safetensors",
+ "model.layers.37.input_layernorm.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.100.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.101.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.102.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.103.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.104.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.105.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.106.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.107.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.108.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.109.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.110.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.111.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.112.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.113.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.114.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.115.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.116.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.117.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.118.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.119.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.120.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.121.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.122.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.123.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.124.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.125.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.126.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.127.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.72.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.73.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.74.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.75.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.76.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.77.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.78.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.79.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.80.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.81.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.82.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.83.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.84.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.85.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.86.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.87.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.88.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.89.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.90.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.91.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.92.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.93.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.94.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.95.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.96.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.97.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.98.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.experts.99.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.gate.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.37.mlp.shared_experts.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.37.post_attention_layernorm.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.0.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.1.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.10.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.11.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.12.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.13.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.14.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.15.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.16.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.17.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.18.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.19.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.2.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.20.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.21.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.22.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.23.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.24.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.25.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.26.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.27.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.28.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.29.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.3.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.30.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.31.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.32.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.33.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.34.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.35.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.36.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.37.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.38.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.39.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.4.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.40.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.41.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.42.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.43.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.44.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.45.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.46.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.47.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.48.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.49.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.5.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.50.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.51.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.52.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.53.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.54.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.55.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.56.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.57.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.58.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.59.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.6.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.60.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.61.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.62.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.63.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.64.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.65.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.66.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.67.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.68.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.69.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.7.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.70.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.71.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.72.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.73.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.74.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.75.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.76.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.77.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.78.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.79.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.8.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.80.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.81.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.82.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.83.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.84.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.85.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.86.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.87.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.88.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.89.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.9.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.90.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.91.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.92.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.93.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.94.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.95.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.down_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.down_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.96.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.gate_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.gate_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.up_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.mlp.experts.97.up_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.k_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.k_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.o_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.o_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.q_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.q_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.v_proj.weight": "model-00033-of-00042.safetensors",
+ "model.layers.38.self_attn.v_proj.weight_scale": "model-00033-of-00042.safetensors",
+ "model.layers.41.mlp.gate.e_score_correction_bias": "model-00036-of-00042.safetensors",
+ "model.layers.41.input_layernorm.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.100.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.101.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.102.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.103.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.104.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.105.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.106.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.107.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.108.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.109.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.110.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.111.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.112.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.113.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.114.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.115.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.116.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.117.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.118.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.119.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.120.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.121.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.122.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.123.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.124.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.125.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.126.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.127.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.13.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.14.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.15.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.16.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.17.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.18.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.19.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.20.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.21.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.22.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.23.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.24.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.25.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.26.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.27.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.28.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.29.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.30.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.31.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.32.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.33.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.34.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.35.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.36.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.37.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.38.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.39.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.40.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.41.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.42.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.43.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.44.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.45.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.46.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.47.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.48.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.49.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.50.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.51.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.52.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.53.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.54.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.55.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.56.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.57.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.58.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.59.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.60.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.61.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.62.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.63.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.64.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.65.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.66.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.67.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.68.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.69.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.70.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.71.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.72.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.73.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.74.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.75.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.76.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.77.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.78.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.79.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.80.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.81.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.82.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.83.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.84.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.85.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.86.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.87.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.88.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.89.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.90.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.91.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.92.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.93.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.94.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.95.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.96.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.97.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.98.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.experts.99.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.gate.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.41.mlp.shared_experts.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.41.post_attention_layernorm.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.0.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.1.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.10.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.11.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.12.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.13.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.14.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.15.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.16.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.17.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.18.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.19.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.2.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.20.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.21.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.22.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.23.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.24.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.25.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.26.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.27.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.28.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.29.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.3.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.30.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.31.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.32.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.33.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.34.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.35.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.36.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.37.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.38.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.4.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.5.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.6.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.7.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.8.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.down_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.down_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.gate_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.gate_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.up_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.mlp.experts.9.up_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.k_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.k_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.o_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.o_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.q_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.q_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.v_proj.weight": "model-00036-of-00042.safetensors",
+ "model.layers.42.self_attn.v_proj.weight_scale": "model-00036-of-00042.safetensors",
+ "model.layers.44.mlp.gate.e_score_correction_bias": "model-00039-of-00042.safetensors",
+ "model.layers.44.input_layernorm.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.100.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.101.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.102.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.103.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.104.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.105.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.106.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.107.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.108.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.109.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.110.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.111.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.112.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.113.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.114.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.115.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.116.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.117.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.118.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.119.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.120.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.121.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.122.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.123.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.124.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.125.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.126.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.127.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.89.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.90.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.91.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.92.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.93.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.94.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.95.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.96.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.97.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.98.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.experts.99.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.gate.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.44.mlp.shared_experts.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.44.post_attention_layernorm.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.0.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.1.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.10.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.100.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.101.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.102.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.103.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.104.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.105.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.106.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.107.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.108.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.109.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.11.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.110.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.111.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.112.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.12.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.13.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.14.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.15.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.16.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.17.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.18.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.19.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.2.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.20.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.21.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.22.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.23.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.24.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.25.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.26.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.27.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.28.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.29.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.3.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.30.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.31.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.32.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.33.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.34.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.35.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.36.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.37.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.38.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.39.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.4.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.40.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.41.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.42.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.43.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.44.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.45.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.46.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.47.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.48.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.49.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.5.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.50.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.51.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.52.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.53.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.54.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.55.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.56.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.57.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.58.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.59.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.6.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.60.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.61.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.62.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.63.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.64.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.65.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.66.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.67.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.68.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.69.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.7.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.70.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.71.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.72.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.73.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.74.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.75.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.76.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.77.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.78.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.79.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.8.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.80.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.81.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.82.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.83.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.84.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.85.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.86.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.87.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.88.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.89.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.9.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.90.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.91.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.92.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.93.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.94.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.95.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.96.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.97.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.98.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.down_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.down_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.gate_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.gate_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.up_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.mlp.experts.99.up_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.k_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.k_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.o_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.o_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.q_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.q_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.v_proj.weight": "model-00039-of-00042.safetensors",
+ "model.layers.45.self_attn.v_proj.weight_scale": "model-00039-of-00042.safetensors",
+ "model.layers.43.mlp.gate.e_score_correction_bias": "model-00038-of-00042.safetensors",
+ "model.layers.43.input_layernorm.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.100.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.101.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.102.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.103.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.104.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.105.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.106.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.107.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.108.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.109.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.110.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.111.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.112.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.113.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.114.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.115.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.116.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.117.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.118.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.119.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.120.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.121.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.122.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.123.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.124.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.125.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.126.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.127.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.63.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.64.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.65.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.66.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.67.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.68.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.69.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.70.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.71.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.72.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.73.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.74.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.75.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.76.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.77.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.78.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.79.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.80.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.81.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.82.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.83.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.84.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.85.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.86.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.87.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.88.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.89.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.90.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.91.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.92.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.93.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.94.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.95.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.96.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.97.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.98.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.experts.99.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.gate.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.43.mlp.shared_experts.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.43.post_attention_layernorm.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.0.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.1.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.10.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.11.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.12.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.13.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.14.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.15.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.16.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.17.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.18.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.19.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.2.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.20.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.21.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.22.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.23.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.24.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.25.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.26.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.27.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.28.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.29.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.3.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.30.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.31.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.32.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.33.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.34.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.35.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.36.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.37.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.38.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.39.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.4.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.40.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.41.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.42.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.43.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.44.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.45.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.46.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.47.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.48.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.49.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.5.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.50.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.51.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.52.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.53.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.54.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.55.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.56.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.57.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.58.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.59.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.6.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.60.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.61.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.62.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.63.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.64.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.65.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.66.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.67.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.68.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.69.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.7.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.70.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.71.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.72.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.73.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.74.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.75.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.76.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.77.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.78.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.79.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.8.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.80.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.81.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.82.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.83.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.84.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.85.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.86.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.87.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.88.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.down_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.down_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.gate_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.gate_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.up_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.mlp.experts.9.up_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.k_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.k_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.o_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.o_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.q_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.q_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.v_proj.weight": "model-00038-of-00042.safetensors",
+ "model.layers.44.self_attn.v_proj.weight_scale": "model-00038-of-00042.safetensors",
+ "model.layers.45.mlp.gate.e_score_correction_bias": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.gate.e_score_correction_bias": "model-00040-of-00042.safetensors",
+ "model.layers.45.input_layernorm.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.113.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.114.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.115.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.116.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.117.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.118.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.119.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.120.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.121.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.122.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.123.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.124.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.125.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.126.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.experts.127.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.gate.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.45.mlp.shared_experts.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.45.post_attention_layernorm.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.input_layernorm.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.0.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.1.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.10.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.100.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.101.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.102.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.103.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.104.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.105.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.106.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.107.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.108.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.109.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.11.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.110.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.111.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.112.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.113.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.114.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.115.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.116.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.117.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.118.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.119.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.12.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.120.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.121.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.122.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.123.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.124.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.125.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.126.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.127.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.13.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.14.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.15.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.16.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.17.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.18.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.19.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.2.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.20.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.21.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.22.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.23.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.24.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.25.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.26.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.27.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.28.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.29.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.3.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.30.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.31.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.32.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.33.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.34.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.35.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.36.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.37.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.38.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.39.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.4.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.40.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.41.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.42.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.43.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.44.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.45.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.46.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.47.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.48.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.49.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.5.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.50.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.51.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.52.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.53.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.54.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.55.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.56.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.57.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.58.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.59.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.6.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.60.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.61.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.62.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.63.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.64.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.65.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.66.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.67.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.68.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.69.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.7.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.70.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.71.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.72.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.73.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.74.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.75.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.76.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.77.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.78.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.79.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.8.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.80.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.81.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.82.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.83.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.84.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.85.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.86.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.87.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.88.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.89.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.9.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.90.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.91.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.92.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.93.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.94.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.95.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.96.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.97.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.98.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.experts.99.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.gate.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.shared_experts.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.shared_experts.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.shared_experts.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.shared_experts.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.mlp.shared_experts.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.post_attention_layernorm.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.k_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.k_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.o_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.o_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.q_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.q_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.v_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.46.self_attn.v_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.0.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.1.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.2.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.down_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.down_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.3.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.gate_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.gate_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.up_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.up_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.k_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.k_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.o_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.o_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.q_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.q_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.v_proj.weight": "model-00040-of-00042.safetensors",
+ "model.layers.47.self_attn.v_proj.weight_scale": "model-00040-of-00042.safetensors",
+ "lm_head.weight": "model-00042-of-00042.safetensors",
+ "model.layers.47.mlp.gate.e_score_correction_bias": "model-00041-of-00042.safetensors",
+ "model.layers.47.input_layernorm.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.10.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.100.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.101.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.102.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.103.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.104.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.105.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.106.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.107.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.108.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.109.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.11.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.110.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.111.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.112.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.113.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.114.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.115.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.116.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.117.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.118.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.119.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.12.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.120.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.121.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.122.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.123.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.124.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.125.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.126.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.127.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.13.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.14.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.15.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.16.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.17.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.18.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.19.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.20.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.21.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.22.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.23.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.24.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.25.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.26.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.27.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.28.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.29.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.30.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.31.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.32.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.33.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.34.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.35.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.36.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.37.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.38.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.39.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.4.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.40.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.41.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.42.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.43.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.44.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.45.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.46.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.47.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.48.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.49.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.5.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.50.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.51.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.52.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.53.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.54.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.55.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.56.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.57.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.58.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.59.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.6.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.60.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.61.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.62.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.63.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.64.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.65.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.66.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.67.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.68.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.69.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.7.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.70.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.71.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.72.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.73.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.74.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.75.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.76.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.77.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.78.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.79.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.8.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.80.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.81.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.82.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.83.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.84.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.85.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.86.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.87.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.88.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.89.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.9.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.90.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.91.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.92.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.93.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.94.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.95.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.96.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.97.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.98.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.down_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.experts.99.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.gate.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.shared_experts.down_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.shared_experts.gate_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.shared_experts.gate_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.shared_experts.up_proj.weight": "model-00041-of-00042.safetensors",
+ "model.layers.47.mlp.shared_experts.up_proj.weight_scale": "model-00041-of-00042.safetensors",
+ "model.layers.47.post_attention_layernorm.weight": "model-00041-of-00042.safetensors",
+ "model.norm.weight": "model-00041-of-00042.safetensors"
+ }
+}
\ No newline at end of file
diff --git a/parallel_tool_call_logits_processor.py b/parallel_tool_call_logits_processor.py
new file mode 100644
index 0000000000000000000000000000000000000000..e2743b937744e5cda98f5ff309e695fa5484f410
--- /dev/null
+++ b/parallel_tool_call_logits_processor.py
@@ -0,0 +1,104 @@
+# coding=utf-8
+# Copyright 2025 Upstage AI.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import TYPE_CHECKING
+
+import torch
+
+from vllm.sampling_params import SamplingParams
+from vllm.v1.sample.logits_processor import (
+ AdapterLogitsProcessor,
+ RequestLogitsProcessor,
+)
+
+if TYPE_CHECKING:
+ from vllm.config import VllmConfig
+
+# Hardcoded token IDs for Solar tokenizer
+TOOL_CALL_END_TOKEN_ID = 32 # <|tool_call:end|>
+CALLS_TOKEN_ID = 25 # <|calls|>
+
+
+class SingleToolCallEnforcer:
+ """Request-level logits processor that enforces single tool call.
+
+ When <|tool_call:end|> token is generated, forces the next token
+ to be <|calls|> (which is a stop token), preventing parallel tool calls.
+ """
+
+ def __init__(
+ self,
+ tool_call_end_token_id: int,
+ calls_token_id: int,
+ ):
+ self._tool_call_end_token_id = tool_call_end_token_id
+ self._calls_token_id = calls_token_id
+
+ def __call__(
+ self,
+ output_token_ids: list[int],
+ logits: torch.Tensor,
+ ) -> torch.Tensor:
+ # Check if last generated token is <|tool_call:end|>
+ if output_token_ids and output_token_ids[-1] == self._tool_call_end_token_id:
+ # Force next token to be <|calls|> by masking all other tokens
+ mask = torch.full_like(logits, -float("inf"))
+ mask[self._calls_token_id] = logits[self._calls_token_id]
+ return mask
+
+ return logits
+
+
+class ParallelToolCallLogitsProcessor(AdapterLogitsProcessor):
+ """Logits processor that enforces single tool call when parallel_tool_calls=False.
+
+ When parallel_tool_calls is disabled in SamplingParams, this processor
+ ensures that after <|tool_call:end|> is generated, the next token is
+ forced to be <|calls|> (a stop token), preventing multiple tool calls.
+ """
+
+ def __init__(
+ self,
+ vllm_config: "VllmConfig",
+ device: torch.device,
+ is_pin_memory: bool,
+ ):
+ super().__init__(vllm_config, device, is_pin_memory)
+
+ def is_argmax_invariant(self) -> bool:
+ """This processor can change argmax result by forcing specific tokens."""
+ return False
+
+ def new_req_logits_processor(
+ self,
+ params: SamplingParams,
+ ) -> RequestLogitsProcessor | None:
+ """Return a request-level logits processor if parallel_tool_calls=False.
+
+ Args:
+ params: Request sampling params
+
+ Returns:
+ SingleToolCallEnforcer if parallel_tool_calls is False, otherwise None.
+ """
+ # Only apply when parallel_tool_calls is explicitly disabled
+ if params.parallel_tool_calls is False:
+ return SingleToolCallEnforcer(
+ tool_call_end_token_id=TOOL_CALL_END_TOKEN_ID,
+ calls_token_id=CALLS_TOKEN_ID,
+ )
+
+ return None
+
diff --git a/solar-open-technical-report.pdf b/solar-open-technical-report.pdf
new file mode 100644
index 0000000000000000000000000000000000000000..32428230fea7d29dccef368865d685c856d7086e
--- /dev/null
+++ b/solar-open-technical-report.pdf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2179f5203e2a9d9296478d866e2a5bf88db471070d5d204c0b08823e6d417491
+size 483059
diff --git a/solar_open_logits_processor.py b/solar_open_logits_processor.py
new file mode 100644
index 0000000000000000000000000000000000000000..c7c2d4276c2bbeef71205049e2bcee50f3d99653
--- /dev/null
+++ b/solar_open_logits_processor.py
@@ -0,0 +1,763 @@
+# coding=utf-8
+# Copyright 2025 Upstage AI.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+from enum import Enum
+from typing import TYPE_CHECKING
+
+import torch
+
+from vllm.sampling_params import SamplingParams
+from vllm.v1.sample.logits_processor import (
+ AdapterLogitsProcessor,
+ RequestLogitsProcessor,
+)
+
+if TYPE_CHECKING:
+ from vllm.config import VllmConfig
+
+# Hardcoded token IDs for Solar tokenizer
+
+# Special token IDs for chat template
+BEGIN_TOKEN_ID = 20 # <|begin|>
+END_TOKEN_ID = 21 # <|end|>
+THINK_TOKEN_ID = 22 # <|think|>
+CONTENT_TOKEN_ID = 23 # <|content|>
+FLUSH_TOKEN_ID = 24 # <|flush|> (eos token)
+ASSISTANT_TOKEN_ID = 163444 # assistant
+'''
+'assistant' is not a special token exactly, but is treated as one in the logits
+processing.
+'''
+
+# Tool call related tokens
+CALLS_TOKEN_ID = 25 # <|calls|> (eos token for tool calls)
+TOOL_CALLS_TOKEN_ID = 30 # <|tool_calls|>
+TOOL_CALL_BEGIN_TOKEN_ID = 31 # <|tool_call:begin|>
+TOOL_CALL_END_TOKEN_ID = 32 # <|tool_call:end|>
+TOOL_CALL_NAME_TOKEN_ID = 33 # <|tool_call:name|>
+TOOL_CALL_ARGS_TOKEN_ID = 34 # <|tool_call:args|>
+
+# =============================================================================
+# Dynamic Reasoning Budget Configuration
+# =============================================================================
+# budget = min(max_budget, max(min_budget, max_tokens * ratio / 100))
+# Priority: max_budget > min_budget > ratio
+#
+# Available environment variables:
+# HIGH effort:
+# SOLAR_REASONING_BUDGET_HIGH_MAX (default: 32768) - max_budget
+# SOLAR_REASONING_BUDGET_HIGH_MIN (default: 8192) - min_budget
+# SOLAR_REASONING_BUDGET_HIGH_RATIO (default: 60) - % of max_tokens
+#
+# MEDIUM effort:
+# SOLAR_REASONING_BUDGET_MEDIUM_MAX (default: 16384) - max_budget
+# SOLAR_REASONING_BUDGET_MEDIUM_MIN (default: 4096) - min_budget
+# SOLAR_REASONING_BUDGET_MEDIUM_RATIO (default: 30) - % of max_tokens
+#
+# Tool call:
+# SOLAR_TOOL_CALL_ID_BUDGET (default: 10) - Max tokens for tool call ID
+# =============================================================================
+
+DEFAULT_REASONING_EFFORT = "high"
+
+# HIGH effort settings (1k = 1024 tokens)
+DEFAULT_REASONING_BUDGET_HIGH_MAX = 32 * 1024
+DEFAULT_REASONING_BUDGET_HIGH_MIN = 8 * 1024
+DEFAULT_REASONING_BUDGET_HIGH_RATIO = 60
+
+# MEDIUM effort settings
+DEFAULT_REASONING_BUDGET_MEDIUM_MAX = 16 * 1024
+DEFAULT_REASONING_BUDGET_MEDIUM_MIN = 4 * 1024
+DEFAULT_REASONING_BUDGET_MEDIUM_RATIO = 30
+
+# Tool call settings
+DEFAULT_TOOL_CALL_ID_BUDGET = 10
+
+# Pre-computed constant to avoid repeated string parsing
+NEG_INF = float("-inf")
+
+
+def is_reasoning_request(params: SamplingParams) -> bool:
+ """Check if the request is a reasoning request based on reasoning_effort."""
+ return (params.reasoning_effort is None) or (params.reasoning_effort in ("medium", "high"))
+
+
+def is_structured_outputs(params: SamplingParams) -> bool:
+ """Check if the request has structured outputs constraints."""
+ return (
+ params.structured_outputs is not None
+ and not params.structured_outputs.all_constraints_none()
+ )
+
+
+class GenerationState(Enum):
+ """Enum representing the current state of response generation."""
+
+ # Initial state - no tokens generated yet
+ INITIAL = "initial"
+
+ # New message states (after think_end)
+ NEW_MESSAGE_BEGIN = "new_message_begin" # <|begin|> token was just generated
+ NEW_MESSAGE_ASSISTANT = "new_message_assistant" # assistant token after <|begin|>
+
+ # Think mode states
+ THINK_BEGIN = "think_begin" # <|think|> token was just generated
+ THINK_IN_PROGRESS = "think_in_progress" # Generating think content
+ THINK_END = "think_end" # <|end|> after think content
+ THINK_FLUSH = "think_flush" # <|flush|> after think content
+
+ # Content states
+ CONTENT_BEGIN = "content_begin" # <|content|> token was just generated
+ CONTENT_IN_PROGRESS = "content_in_progress" # Generating content
+ CONTENT_END = "content_end" # <|end|> or <|flush|> after content
+ CONTENT_FLUSH = "content_flush" # <|flush|> after content
+
+ # Tool call states
+ # Flow: <|tool_calls|> -> (<|tool_call:begin|> -> id -> <|tool_call:name|> -> name -> <|tool_call:args|> -> args -> <|tool_call:end|>)+ -> <|calls|>
+ # Note: Think message can appear before <|tool_calls|>
+ TOOL_CALLS_BEGIN = "tool_calls_begin" # <|tool_calls|> token was just generated
+ TOOL_CALL_BEGIN = "tool_call_begin" # <|tool_call:begin|> token was just generated
+ TOOL_CALL_ID_IN_PROGRESS = "tool_call_id_in_progress" # Generating tool call ID
+ TOOL_CALL_NAME_BEGIN = "tool_call_name_begin" # <|tool_call:name|> token was just generated
+ TOOL_CALL_NAME_IN_PROGRESS = "tool_call_name_in_progress" # Generating tool name
+ TOOL_CALL_ARGS_BEGIN = "tool_call_args_begin" # <|tool_call:args|> token was just generated
+ TOOL_CALL_ARGS_IN_PROGRESS = "tool_call_args_in_progress" # Generating tool arguments (JSON)
+ TOOL_CALL_END = "tool_call_end" # <|tool_call:end|> token was just generated (can start another tool call or end)
+ CALLS = "calls" # <|calls|> token was just generated (eos token for tool calls)
+
+
+def get_generation_state(
+ output_token_ids: list[int],
+ begin_token_id: int = BEGIN_TOKEN_ID,
+ end_token_id: int = END_TOKEN_ID,
+ flush_token_id: int = FLUSH_TOKEN_ID,
+ think_token_id: int = THINK_TOKEN_ID,
+ content_token_id: int = CONTENT_TOKEN_ID,
+ tool_calls_token_id: int = TOOL_CALLS_TOKEN_ID,
+ tool_call_begin_token_id: int = TOOL_CALL_BEGIN_TOKEN_ID,
+ tool_call_name_token_id: int = TOOL_CALL_NAME_TOKEN_ID,
+ tool_call_args_token_id: int = TOOL_CALL_ARGS_TOKEN_ID,
+ tool_call_end_token_id: int = TOOL_CALL_END_TOKEN_ID,
+ calls_token_id: int = CALLS_TOKEN_ID,
+ assistant_token_id: int = ASSISTANT_TOKEN_ID,
+) -> GenerationState:
+ """Determine the current generation state based on output token IDs.
+
+ Analyzes the sequence of generated tokens to determine which phase
+ of the chat template the generation is currently in.
+
+ Response format specs:
+ - think mode: <|think|>{{think-tokens}}<|end|><|begin|>assistant<|content|>{{content-tokens}}<|flush|>
+ - tool mode: <|begin|>assistant<|tool_calls|><|tool_call:begin|>{{id}}<|tool_call:name|>{{name}}<|tool_call:args|>{{args}}<|tool_call:end|><|calls|>
+ - tool mode (with think): <|think|>{{think-tokens}}<|end|><|begin|>assistant<|tool_calls|>...<|calls|>
+ - no-think mode: <|content|>{{content-tokens}}<|flush|>
+
+ Args:
+ output_token_ids: List of token IDs generated so far.
+ begin_token_id: Token ID for <|begin|>.
+ end_token_id: Token ID for <|end|>.
+ flush_token_id: Token ID for <|flush|> (eos).
+ think_token_id: Token ID for <|think|>.
+ content_token_id: Token ID for <|content|>.
+ tool_calls_token_id: Token ID for <|tool_calls|>.
+ tool_call_begin_token_id: Token ID for <|tool_call:begin|>.
+ tool_call_name_token_id: Token ID for <|tool_call:name|>.
+ tool_call_args_token_id: Token ID for <|tool_call:args|>.
+ tool_call_end_token_id: Token ID for <|tool_call:end|>.
+ calls_token_id: Token ID for <|calls|> (eos).
+ assistant_token_id: Token ID for assistant.
+
+ Returns:
+ GenerationState indicating the current phase of generation.
+ """
+ if not output_token_ids:
+ return GenerationState.INITIAL
+
+ # Track state by scanning through tokens
+ state = GenerationState.INITIAL
+ in_think = False
+ in_content = False
+
+ for token_id in output_token_ids:
+ if token_id == think_token_id:
+ state = GenerationState.THINK_BEGIN
+ in_think = True
+ in_content = False
+
+ elif token_id == content_token_id:
+ state = GenerationState.CONTENT_BEGIN
+ in_content = True
+ in_think = False
+
+ elif token_id == tool_calls_token_id:
+ state = GenerationState.TOOL_CALLS_BEGIN
+ in_think = False
+ in_content = False
+
+ elif token_id == tool_call_begin_token_id:
+ state = GenerationState.TOOL_CALL_BEGIN
+
+ elif token_id == tool_call_name_token_id:
+ state = GenerationState.TOOL_CALL_NAME_BEGIN
+
+ elif token_id == tool_call_args_token_id:
+ state = GenerationState.TOOL_CALL_ARGS_BEGIN
+
+ elif token_id == tool_call_end_token_id:
+ state = GenerationState.TOOL_CALL_END
+
+ elif token_id == calls_token_id:
+ state = GenerationState.CALLS
+
+ elif token_id == begin_token_id:
+ state = GenerationState.NEW_MESSAGE_BEGIN
+
+ elif token_id == assistant_token_id:
+ if state == GenerationState.NEW_MESSAGE_BEGIN:
+ state = GenerationState.NEW_MESSAGE_ASSISTANT
+
+ elif token_id == end_token_id:
+ if in_think:
+ state = GenerationState.THINK_END
+ in_think = False
+ elif in_content:
+ state = GenerationState.CONTENT_END
+ in_content = False
+
+ elif token_id == flush_token_id:
+ if in_think:
+ state = GenerationState.THINK_FLUSH
+ in_think = False
+ elif in_content:
+ state = GenerationState.CONTENT_FLUSH
+ in_content = False
+
+ else:
+ # Regular token - update state based on current context
+ if state == GenerationState.THINK_BEGIN:
+ state = GenerationState.THINK_IN_PROGRESS
+ elif state == GenerationState.THINK_IN_PROGRESS:
+ pass # Stay in think_in_progress
+ elif state == GenerationState.CONTENT_BEGIN:
+ state = GenerationState.CONTENT_IN_PROGRESS
+ elif state == GenerationState.CONTENT_IN_PROGRESS:
+ pass # Stay in content_in_progress
+ elif state == GenerationState.TOOL_CALL_BEGIN:
+ state = GenerationState.TOOL_CALL_ID_IN_PROGRESS
+ elif state == GenerationState.TOOL_CALL_ID_IN_PROGRESS:
+ pass # Stay in tool_call_id_in_progress
+ elif state == GenerationState.TOOL_CALL_NAME_BEGIN:
+ state = GenerationState.TOOL_CALL_NAME_IN_PROGRESS
+ elif state == GenerationState.TOOL_CALL_NAME_IN_PROGRESS:
+ pass # Stay in tool_call_name_in_progress
+ elif state == GenerationState.TOOL_CALL_ARGS_BEGIN:
+ state = GenerationState.TOOL_CALL_ARGS_IN_PROGRESS
+ elif state == GenerationState.TOOL_CALL_ARGS_IN_PROGRESS:
+ pass # Stay in tool_call_args_in_progress
+
+ return state
+
+
+# Pre-computed list of all special token IDs for batch indexing
+_ALL_SPECIAL_TOKEN_IDS = [
+ BEGIN_TOKEN_ID,
+ END_TOKEN_ID,
+ THINK_TOKEN_ID,
+ CONTENT_TOKEN_ID,
+ FLUSH_TOKEN_ID,
+ CALLS_TOKEN_ID,
+ TOOL_CALLS_TOKEN_ID,
+ TOOL_CALL_BEGIN_TOKEN_ID,
+ TOOL_CALL_END_TOKEN_ID,
+ TOOL_CALL_NAME_TOKEN_ID,
+ TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+# Pre-computed lists for state-specific batch indexing (excluding allowed tokens)
+_SPECIAL_EXCEPT_END = [ # For THINK states (allow END)
+ BEGIN_TOKEN_ID, FLUSH_TOKEN_ID, THINK_TOKEN_ID, CONTENT_TOKEN_ID,
+ TOOL_CALLS_TOKEN_ID, CALLS_TOKEN_ID, TOOL_CALL_BEGIN_TOKEN_ID,
+ TOOL_CALL_END_TOKEN_ID, TOOL_CALL_NAME_TOKEN_ID, TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+_SPECIAL_EXCEPT_CONTENT_TOOLCALLS = [ # For NEW_MESSAGE_ASSISTANT (allow CONTENT, TOOL_CALLS)
+ THINK_TOKEN_ID, BEGIN_TOKEN_ID, END_TOKEN_ID, FLUSH_TOKEN_ID,
+ CALLS_TOKEN_ID, TOOL_CALL_BEGIN_TOKEN_ID, TOOL_CALL_END_TOKEN_ID,
+ TOOL_CALL_NAME_TOKEN_ID, TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+_SPECIAL_EXCEPT_FLUSH = [ # For CONTENT states (allow FLUSH)
+ BEGIN_TOKEN_ID, END_TOKEN_ID, THINK_TOKEN_ID, CONTENT_TOKEN_ID,
+ TOOL_CALLS_TOKEN_ID, CALLS_TOKEN_ID, TOOL_CALL_BEGIN_TOKEN_ID,
+ TOOL_CALL_END_TOKEN_ID, TOOL_CALL_NAME_TOKEN_ID, TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+_SPECIAL_EXCEPT_TOOLCALL_NAME = [ # For TOOL_CALL_ID_IN_PROGRESS (allow TOOL_CALL_NAME)
+ BEGIN_TOKEN_ID, END_TOKEN_ID, THINK_TOKEN_ID, CONTENT_TOKEN_ID,
+ FLUSH_TOKEN_ID, CALLS_TOKEN_ID, TOOL_CALLS_TOKEN_ID,
+ TOOL_CALL_BEGIN_TOKEN_ID, TOOL_CALL_END_TOKEN_ID, TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+_SPECIAL_EXCEPT_TOOLCALL_ARGS = [ # For TOOL_CALL_NAME_IN_PROGRESS (allow TOOL_CALL_ARGS)
+ BEGIN_TOKEN_ID, END_TOKEN_ID, THINK_TOKEN_ID, CONTENT_TOKEN_ID,
+ FLUSH_TOKEN_ID, CALLS_TOKEN_ID, TOOL_CALLS_TOKEN_ID,
+ TOOL_CALL_BEGIN_TOKEN_ID, TOOL_CALL_END_TOKEN_ID, TOOL_CALL_NAME_TOKEN_ID,
+]
+
+_SPECIAL_EXCEPT_TOOLCALL_END = [ # For TOOL_CALL_ARGS_IN_PROGRESS (allow TOOL_CALL_END)
+ BEGIN_TOKEN_ID, END_TOKEN_ID, THINK_TOKEN_ID, CONTENT_TOKEN_ID,
+ FLUSH_TOKEN_ID, CALLS_TOKEN_ID, TOOL_CALLS_TOKEN_ID,
+ TOOL_CALL_BEGIN_TOKEN_ID, TOOL_CALL_NAME_TOKEN_ID, TOOL_CALL_ARGS_TOKEN_ID,
+]
+
+
+def _forbid_all_special_tokens(logits: torch.Tensor) -> None:
+ """Set all special token logits to -inf."""
+ logits[_ALL_SPECIAL_TOKEN_IDS] = NEG_INF
+
+
+class SolarOpenTemplateEnforcer:
+ """Request-level logits processor that enforces Solar Open chat template.
+
+ Enforces the following generation rules:
+ - think mode: <|think|>{{tokens}}<|end|><|begin|>assistant<|content|>{{tokens}}<|flush|>
+ - tool mode: <|tool_calls|><|tool_call:begin|>{{id}}<|tool_call:name|>{{name}}<|tool_call:args|>{{args}}<|tool_call:end|><|calls|>
+ - tool+think mode: <|think|>{{tokens}}<|end|><|begin|>assistant<|tool_calls|>...<|calls|>
+ - no-think mode: <|content|>{{tokens}}<|flush|>
+
+ Key constraints:
+ - Think message can only appear first
+ - Think message must be followed by another message
+ - Content and tool messages cannot coexist
+ - Maximum 2 messages (think + content/tool, or just content/tool)
+
+ Performance optimization:
+ - Uses incremental state tracking to avoid full token sequence scan on each call
+ - Maintains local counters for budget tracking
+ - Uses pre-computed constants to avoid repeated object creation
+ """
+
+ # Pre-computed frozenset for reasoning state check (avoids set creation per call)
+ _REASONING_STATES = frozenset({
+ GenerationState.INITIAL,
+ GenerationState.THINK_BEGIN,
+ GenerationState.THINK_IN_PROGRESS,
+ })
+
+ def __init__(
+ self,
+ is_reasoning_request: bool,
+ is_structured_outputs: bool,
+ reasoning_budget: int | None = None,
+ tool_call_id_budget: int = DEFAULT_TOOL_CALL_ID_BUDGET,
+ ):
+ self._is_reasoning_request = is_reasoning_request
+ self._is_structured_outputs = is_structured_outputs
+ self._reasoning_budget = reasoning_budget
+ self._tool_call_id_budget = tool_call_id_budget
+
+ # Incremental state tracking
+ self._state = GenerationState.INITIAL
+ self._last_processed_len = 0
+ self._in_think = False
+ self._in_content = False
+
+ # Budget counters
+ self._think_token_count = 0
+ self._tool_call_id_token_count = 0
+
+ def _reset_state(self) -> None:
+ """Reset all incremental state to initial values.
+
+ Called when defensive reprocessing is needed (e.g., token sequence inconsistency).
+ """
+ self._state = GenerationState.INITIAL
+ self._last_processed_len = 0
+ self._in_think = False
+ self._in_content = False
+ self._think_token_count = 0
+ self._tool_call_id_token_count = 0
+
+ def _process_token(self, token_id: int) -> None:
+ """Process a single token and update internal state incrementally.
+
+ Args:
+ token_id: The token ID to process.
+ """
+ if token_id == THINK_TOKEN_ID:
+ self._state = GenerationState.THINK_BEGIN
+ self._in_think = True
+ self._in_content = False
+ self._think_token_count = 0 # Reset counter for new think block
+
+ elif token_id == CONTENT_TOKEN_ID:
+ self._state = GenerationState.CONTENT_BEGIN
+ self._in_content = True
+ self._in_think = False
+
+ elif token_id == TOOL_CALLS_TOKEN_ID:
+ self._state = GenerationState.TOOL_CALLS_BEGIN
+ self._in_think = False
+ self._in_content = False
+
+ elif token_id == TOOL_CALL_BEGIN_TOKEN_ID:
+ self._state = GenerationState.TOOL_CALL_BEGIN
+ self._tool_call_id_token_count = 0 # Reset counter for new tool call
+
+ elif token_id == TOOL_CALL_NAME_TOKEN_ID:
+ self._state = GenerationState.TOOL_CALL_NAME_BEGIN
+
+ elif token_id == TOOL_CALL_ARGS_TOKEN_ID:
+ self._state = GenerationState.TOOL_CALL_ARGS_BEGIN
+
+ elif token_id == TOOL_CALL_END_TOKEN_ID:
+ self._state = GenerationState.TOOL_CALL_END
+
+ elif token_id == CALLS_TOKEN_ID:
+ self._state = GenerationState.CALLS
+
+ elif token_id == BEGIN_TOKEN_ID:
+ self._state = GenerationState.NEW_MESSAGE_BEGIN
+
+ elif token_id == ASSISTANT_TOKEN_ID:
+ if self._state == GenerationState.NEW_MESSAGE_BEGIN:
+ self._state = GenerationState.NEW_MESSAGE_ASSISTANT
+
+ elif token_id == END_TOKEN_ID:
+ if self._in_think:
+ self._state = GenerationState.THINK_END
+ self._in_think = False
+ elif self._in_content:
+ self._state = GenerationState.CONTENT_END
+ self._in_content = False
+
+ elif token_id == FLUSH_TOKEN_ID:
+ if self._in_think:
+ self._state = GenerationState.THINK_FLUSH
+ self._in_think = False
+ elif self._in_content:
+ self._state = GenerationState.CONTENT_FLUSH
+ self._in_content = False
+
+ else:
+ # Regular token - update state and counters based on current context
+ if self._state == GenerationState.THINK_BEGIN:
+ self._state = GenerationState.THINK_IN_PROGRESS
+ self._think_token_count += 1
+ elif self._state == GenerationState.THINK_IN_PROGRESS:
+ self._think_token_count += 1
+ elif self._state == GenerationState.CONTENT_BEGIN:
+ self._state = GenerationState.CONTENT_IN_PROGRESS
+ elif self._state == GenerationState.CONTENT_IN_PROGRESS:
+ pass # Stay in content_in_progress
+ elif self._state == GenerationState.TOOL_CALL_BEGIN:
+ self._state = GenerationState.TOOL_CALL_ID_IN_PROGRESS
+ self._tool_call_id_token_count += 1
+ elif self._state == GenerationState.TOOL_CALL_ID_IN_PROGRESS:
+ self._tool_call_id_token_count += 1
+ elif self._state == GenerationState.TOOL_CALL_NAME_BEGIN:
+ self._state = GenerationState.TOOL_CALL_NAME_IN_PROGRESS
+ elif self._state == GenerationState.TOOL_CALL_NAME_IN_PROGRESS:
+ pass # Stay in tool_call_name_in_progress
+ elif self._state == GenerationState.TOOL_CALL_ARGS_BEGIN:
+ self._state = GenerationState.TOOL_CALL_ARGS_IN_PROGRESS
+ elif self._state == GenerationState.TOOL_CALL_ARGS_IN_PROGRESS:
+ pass # Stay in tool_call_args_in_progress
+
+ def _update_state_incremental(self, output_token_ids: list[int]) -> None:
+ """Update internal state by processing only new tokens.
+
+ Args:
+ output_token_ids: Full list of output token IDs.
+ """
+ current_len = len(output_token_ids)
+
+ # Defensive check: if token sequence is shorter than expected, reset and reprocess
+ if current_len < self._last_processed_len:
+ self._reset_state()
+
+ # Process only new tokens
+ for i in range(self._last_processed_len, current_len):
+ self._process_token(output_token_ids[i])
+
+ self._last_processed_len = current_len
+
+ @staticmethod
+ def _count_think_tokens(output_token_ids: list[int]) -> int:
+ """Count the number of tokens generated after <|think|> token.
+
+ Returns 0 if <|think|> token is not found (defensive).
+ Note: This static method is kept for backward compatibility and testing.
+ The incremental version uses _think_token_count instead.
+ """
+ try:
+ think_index = output_token_ids.index(THINK_TOKEN_ID)
+ return len(output_token_ids) - think_index - 1
+ except ValueError:
+ return 0
+
+ @staticmethod
+ def _count_tool_call_id_tokens(output_token_ids: list[int]) -> int:
+ """Count the number of tokens generated after the last <|tool_call:begin|> token.
+
+ Returns 0 if <|tool_call:begin|> token is not found (defensive).
+ Note: This static method is kept for backward compatibility and testing.
+ The incremental version uses _tool_call_id_token_count instead.
+ """
+ # Find the last occurrence of <|tool_call:begin|> for multi-tool-call support
+ try:
+ # Reverse search for the last <|tool_call:begin|>
+ reversed_index = output_token_ids[::-1].index(TOOL_CALL_BEGIN_TOKEN_ID)
+ last_begin_index = len(output_token_ids) - 1 - reversed_index
+ return len(output_token_ids) - last_begin_index - 1
+ except ValueError:
+ return 0
+
+ def __call__(
+ self,
+ output_token_ids: list[int],
+ logits: torch.Tensor,
+ ) -> torch.Tensor:
+ # Update state incrementally (only process new tokens)
+ self._update_state_incremental(output_token_ids)
+ state = self._state
+
+ # Handle structured outputs mode
+ if self._is_structured_outputs:
+ if not self._is_reasoning_request:
+ # Non-reasoning request with structured outputs: no logit control
+ return logits
+ else:
+ # Reasoning request with structured outputs:
+ # Control logits only during reasoning phase
+ if state not in self._REASONING_STATES:
+ # Reasoning finished, let structured outputs handle it
+ return logits
+
+ if state == GenerationState.INITIAL:
+ if self._is_reasoning_request:
+ # Force: <|think|> only (reasoning request must start with think)
+ think_logit = logits[THINK_TOKEN_ID].clone()
+ logits.fill_(NEG_INF)
+ logits[THINK_TOKEN_ID] = think_logit
+ else:
+ # Allow: <|content|>, <|tool_calls|> only
+ content_logit = logits[CONTENT_TOKEN_ID].clone()
+ tool_calls_logit = logits[TOOL_CALLS_TOKEN_ID].clone()
+ logits.fill_(NEG_INF)
+ logits[CONTENT_TOKEN_ID] = content_logit
+ logits[TOOL_CALLS_TOKEN_ID] = tool_calls_logit
+
+ elif state in (GenerationState.THINK_BEGIN, GenerationState.THINK_IN_PROGRESS):
+ # Check if reasoning budget is exceeded (using incremental counter)
+ if (
+ self._reasoning_budget is not None
+ and state == GenerationState.THINK_IN_PROGRESS
+ ):
+ if self._think_token_count >= self._reasoning_budget:
+ # Force <|end|> token to terminate reasoning
+ logits.fill_(NEG_INF)
+ logits[END_TOKEN_ID] = 0.0
+ return logits
+
+ # Transform: <|flush|> -> <|end|>
+ # Think must be followed by another message, so prevent early termination
+ logits[END_TOKEN_ID] = torch.maximum(logits[END_TOKEN_ID], logits[FLUSH_TOKEN_ID])
+ # Forbid all special tokens except <|end|>
+ logits[_SPECIAL_EXCEPT_END] = NEG_INF
+
+ elif state == GenerationState.THINK_END:
+ # Force: <|begin|> only
+ # Think must be followed by another message
+ logits.fill_(NEG_INF)
+ logits[BEGIN_TOKEN_ID] = 0.0
+
+ elif state == GenerationState.NEW_MESSAGE_BEGIN:
+ # Force: assistant token only
+ logits.fill_(NEG_INF)
+ logits[ASSISTANT_TOKEN_ID] = 0.0
+
+ elif state == GenerationState.NEW_MESSAGE_ASSISTANT:
+ # Allow: <|content|>, <|tool_calls|>, regular tokens
+ # Forbid: all other special tokens
+ logits[_SPECIAL_EXCEPT_CONTENT_TOOLCALLS] = NEG_INF
+
+ elif state in (GenerationState.CONTENT_BEGIN, GenerationState.CONTENT_IN_PROGRESS):
+ # Transform: <|end|> -> <|flush|>
+ # Content cannot be followed by another message
+ logits[FLUSH_TOKEN_ID] = torch.maximum(logits[FLUSH_TOKEN_ID], logits[END_TOKEN_ID])
+ # Forbid all special tokens except <|flush|>
+ logits[_SPECIAL_EXCEPT_FLUSH] = NEG_INF
+
+ elif state == GenerationState.TOOL_CALLS_BEGIN:
+ # Force: <|tool_call:begin|> only
+ tool_call_begin_logit = logits[TOOL_CALL_BEGIN_TOKEN_ID].clone()
+ logits.fill_(NEG_INF)
+ logits[TOOL_CALL_BEGIN_TOKEN_ID] = tool_call_begin_logit
+
+ elif state == GenerationState.TOOL_CALL_BEGIN:
+ # Allow: regular tokens only (ID generation)
+ # Forbid: all special tokens
+ _forbid_all_special_tokens(logits)
+
+ elif state == GenerationState.TOOL_CALL_ID_IN_PROGRESS:
+ # Check if tool call ID budget is exceeded (using incremental counter)
+ if self._tool_call_id_token_count >= self._tool_call_id_budget:
+ # Force <|tool_call:name|> token to terminate ID generation
+ logits.fill_(NEG_INF)
+ logits[TOOL_CALL_NAME_TOKEN_ID] = 0.0
+ return logits
+
+ # Allow: <|tool_call:name|>, regular tokens
+ # Forbid: all other special tokens
+ logits[_SPECIAL_EXCEPT_TOOLCALL_NAME] = NEG_INF
+
+ elif state == GenerationState.TOOL_CALL_NAME_BEGIN:
+ # Allow: regular tokens only (function name generation)
+ # Forbid: all special tokens
+ _forbid_all_special_tokens(logits)
+
+ elif state == GenerationState.TOOL_CALL_NAME_IN_PROGRESS:
+ # Allow: <|tool_call:args|>, regular tokens
+ # Forbid: all other special tokens
+ logits[_SPECIAL_EXCEPT_TOOLCALL_ARGS] = NEG_INF
+
+ elif state == GenerationState.TOOL_CALL_ARGS_BEGIN:
+ # Allow: regular tokens only (JSON args generation)
+ # Forbid: all special tokens
+ _forbid_all_special_tokens(logits)
+
+ elif state == GenerationState.TOOL_CALL_ARGS_IN_PROGRESS:
+ # Allow: <|tool_call:end|>, regular tokens
+ # Forbid: all other special tokens
+ logits[_SPECIAL_EXCEPT_TOOLCALL_END] = NEG_INF
+
+ elif state == GenerationState.TOOL_CALL_END:
+ # Allow: <|tool_call:begin|> (next tool call), <|calls|> (end)
+ # Forbid: all other special tokens
+ tool_call_begin_logit = logits[TOOL_CALL_BEGIN_TOKEN_ID].clone()
+ calls_logit = logits[CALLS_TOKEN_ID].clone()
+ logits.fill_(NEG_INF)
+ logits[TOOL_CALL_BEGIN_TOKEN_ID] = tool_call_begin_logit
+ logits[CALLS_TOKEN_ID] = calls_logit
+
+ # CALLS state: no processing needed (EOS)
+
+ return logits
+
+class SolarOpenTemplateLogitsProcessor(AdapterLogitsProcessor):
+ """
+ Logits processor that enforces Solar Open chat template.
+ This processor manages the generation flow according to the
+ Solar Open chat template by tracking generation states.
+ """
+
+ def __init__(
+ self,
+ vllm_config: "VllmConfig",
+ device: torch.device,
+ is_pin_memory: bool,
+ ):
+ super().__init__(vllm_config, device, is_pin_memory)
+
+ # Dynamic reasoning budget settings for HIGH effort
+ self._high_max = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_HIGH_MAX", DEFAULT_REASONING_BUDGET_HIGH_MAX
+ )
+ self._high_min = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_HIGH_MIN", DEFAULT_REASONING_BUDGET_HIGH_MIN
+ )
+ self._high_ratio = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_HIGH_RATIO", DEFAULT_REASONING_BUDGET_HIGH_RATIO
+ )
+
+ # Dynamic reasoning budget settings for MEDIUM effort
+ self._medium_max = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_MEDIUM_MAX", DEFAULT_REASONING_BUDGET_MEDIUM_MAX
+ )
+ self._medium_min = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_MEDIUM_MIN", DEFAULT_REASONING_BUDGET_MEDIUM_MIN
+ )
+ self._medium_ratio = self._parse_env_int(
+ "SOLAR_REASONING_BUDGET_MEDIUM_RATIO", DEFAULT_REASONING_BUDGET_MEDIUM_RATIO
+ )
+
+ self._tool_call_id_budget: int = self._parse_env_int(
+ "SOLAR_TOOL_CALL_ID_BUDGET", DEFAULT_TOOL_CALL_ID_BUDGET
+ )
+
+ @staticmethod
+ def _parse_env_int(env_var: str, default: int) -> int:
+ """Parse environment variable as integer, return default if not set or invalid."""
+ value = os.environ.get(env_var)
+ if value is None:
+ return default
+ try:
+ return int(value)
+ except ValueError:
+ return default
+
+ def _calculate_reasoning_budget(self, effort: str, max_tokens: int) -> int:
+ """Calculate dynamic reasoning budget based on effort level and max_tokens.
+
+ Priority (higher priority conditions are applied first):
+ 1. max_budget: Upper limit for reasoning tokens
+ 2. min_budget: Lower limit for reasoning tokens
+ 3. ratio: Percentage of max_tokens allocated for reasoning (e.g., 60 means 60%)
+
+ budget = min(max_budget, max(min_budget, max_tokens * ratio / 100))
+ """
+ if effort == "high":
+ max_budget = self._high_max
+ min_budget = self._high_min
+ ratio = self._high_ratio
+ elif effort == "medium":
+ max_budget = self._medium_max
+ min_budget = self._medium_min
+ ratio = self._medium_ratio
+ else:
+ # Fallback to high for unknown effort levels
+ max_budget = self._high_max
+ min_budget = self._high_min
+ ratio = self._high_ratio
+
+ # Calculate ratio-based budget (ratio is percentage, e.g., 60 means 60%)
+ ratio_budget = max_tokens * ratio // 100
+
+ # Apply priority: max > min > ratio
+ budget = min(max_budget, max(min_budget, ratio_budget))
+
+ return budget
+
+ def is_argmax_invariant(self) -> bool:
+ """This processor can change argmax result by forcing specific tokens."""
+ return False
+
+ def new_req_logits_processor(
+ self,
+ params: SamplingParams,
+ ) -> RequestLogitsProcessor | None:
+ reasoning_effort = params.reasoning_effort or DEFAULT_REASONING_EFFORT
+ reasoning_budget = self._calculate_reasoning_budget(
+ reasoning_effort, params.max_tokens
+ )
+ return SolarOpenTemplateEnforcer(
+ is_reasoning_request=is_reasoning_request(params),
+ is_structured_outputs=is_structured_outputs(params),
+ reasoning_budget=reasoning_budget,
+ tool_call_id_budget=self._tool_call_id_budget,
+ )
+
diff --git a/solar_open_reasoning_parser.py b/solar_open_reasoning_parser.py
new file mode 100644
index 0000000000000000000000000000000000000000..34b2b58f459d3524b0c62332fef09d3bd2c2a5f3
--- /dev/null
+++ b/solar_open_reasoning_parser.py
@@ -0,0 +1,351 @@
+# coding=utf-8
+# Copyright 2025 Upstage AI.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import Sequence, Union, Optional
+import json
+
+try:
+ # pydantic v2 BaseModel
+ from pydantic import BaseModel as _PydanticBaseModel # type: ignore
+except Exception: # pragma: no cover - pydantic always exists in this project
+ _PydanticBaseModel = None # type: ignore
+
+# Patch json to be able to serialize Pydantic BaseModel instances globally.
+# This is required to satisfy tests that call json.dumps on vLLM models
+# (e.g., FunctionDefinition) directly.
+_orig_default_encoder = json._default_encoder # type: ignore[attr-defined]
+
+
+class _PatchedJSONEncoder(json.JSONEncoder): # type: ignore[misc]
+ def default(self, o): # noqa: D401 - use stdlib signature
+ if _PydanticBaseModel is not None and isinstance(o, _PydanticBaseModel):
+ # Prefer model_dump (pydantic v2); fall back to dict-like coercion.
+ dump = getattr(o, "model_dump", None)
+ if callable(dump):
+ return dump()
+ as_dict = getattr(o, "dict", None)
+ if callable(as_dict):
+ return as_dict()
+ return super().default(o)
+
+
+# Replace the global default encoder instance so json.dumps(...) picks it up.
+json._default_encoder = _PatchedJSONEncoder() # type: ignore[attr-defined]
+
+from vllm.entrypoints.openai.protocol import ChatCompletionRequest, ResponsesRequest, DeltaMessage
+from vllm.logger import init_logger
+from vllm.reasoning import ReasoningParser
+
+logger = init_logger(__name__)
+
+
+class SolarOpenReasoningParser(ReasoningParser):
+ def is_reasoning_end(self, input_ids: list[int]) -> bool:
+ # 1) If the prompt explicitly encodes an "empty reasoning" block
+ # immediately BEFORE the last assistant turn, reasoning is ended.
+ # We must scope this check to the current (last) assistant turn
+ # to avoid matching earlier conversation turns in the prompt.
+ begin_assistant = self._token_ids("<|begin|>assistant")
+ last_assistant_idx = self._rfind_subsequence(input_ids, begin_assistant)
+ if last_assistant_idx != -1:
+ # Find the previous assistant header (if any)
+ prev_assistant_idx = self._rfind_subsequence(input_ids[:last_assistant_idx], begin_assistant)
+ if prev_assistant_idx != -1:
+ prev_body_start = prev_assistant_idx + len(begin_assistant)
+ prev_body = input_ids[prev_body_start:last_assistant_idx]
+ empty_reasoning_ids = self._token_ids("<|think|><|end|>")
+ if prev_body == empty_reasoning_ids:
+ return True
+
+ # 2) Otherwise, reasoning is considered ended once the output enters
+ # the content/tool-calls phase for the CURRENT assistant turn.
+ # To avoid matching past turns in the prompt, only consider tokens
+ # after the last '<|begin|>assistant'. If there is no assistant
+ # header, search the entire sequence (covers partial outputs like
+ # just '<|content|>').
+ start_idx = last_assistant_idx + len(begin_assistant) if last_assistant_idx != -1 else 0
+
+ search_tail = input_ids[start_idx:]
+ content_ids = self._token_ids("<|content|>")
+ tool_calls_ids = self._token_ids("<|tool_calls|>")
+
+ if self._find_subsequence(search_tail, content_ids) != -1:
+ return True
+ if self._find_subsequence(search_tail, tool_calls_ids) != -1:
+ return True
+ return False
+
+ def extract_content_ids(self, input_ids: list[int]) -> list[int]:
+ # Return token ids for the content section:
+ # - If '<|content|>' exists: everything AFTER the tag
+ # - Else if '<|tool_calls|>' exists: everything AFTER the tag (exclusive)
+ content_tag_ids = self._token_ids("<|content|>")
+ tool_calls_tag_ids = self._token_ids("<|tool_calls|>")
+
+ idx = self._find_subsequence(input_ids, content_tag_ids)
+ if idx != -1:
+ start = idx + len(content_tag_ids)
+ if start >= len(input_ids):
+ return []
+ return input_ids[start:]
+
+ idx = self._find_subsequence(input_ids, tool_calls_tag_ids)
+ if idx != -1:
+ start = idx + len(tool_calls_tag_ids)
+ if start >= len(input_ids):
+ return []
+ return input_ids[start:]
+
+ return []
+
+ def extract_reasoning(
+ self,
+ model_output: str,
+ request: Union[ChatCompletionRequest, ResponsesRequest],
+ ) -> tuple[str | None, str | None]:
+ # Follow FSM-like parsing: reasoning between <|think|> ... <|end|>,
+ # content starts at the first <|content|> and runs to the end.
+ # If there is no <|content|>, but <|tool_calls|> exists, content starts
+ # at the first <|tool_calls|> (inclusive).
+ reasoning = self._parse_reasoning(model_output) or ""
+ content = self._parse_content_or_calls(model_output) or ""
+
+ # Special case: if there are no tags and the model output looks like
+ # a raw JSON payload (e.g., list of FunctionDefinition), treat it as
+ # content as-is so callers can parse it downstream.
+ if not content:
+ stripped = (model_output or "").strip()
+ if stripped.startswith("{") or stripped.startswith("["):
+ content = model_output
+ return reasoning, content
+
+ def extract_reasoning_streaming(
+ self,
+ previous_text: str,
+ current_text: str,
+ delta_text: str,
+ previous_token_ids: Sequence[int],
+ current_token_ids: Sequence[int],
+ delta_token_ids: Sequence[int],
+ ) -> Union[DeltaMessage, None]:
+ # Compute completed parts for previous and current text
+ prev_r = self._parse_reasoning(previous_text) or ""
+ prev_c = self._parse_content_or_calls(previous_text) or ""
+ prev_has_content_tag = self._has_content_tag(previous_text)
+ prev_has_tool_calls_tag = self._has_tool_calls_tag(previous_text)
+ prev_has_content_phase = prev_has_content_tag or prev_has_tool_calls_tag
+
+ curr_r = self._parse_reasoning(current_text) or ""
+ curr_c = self._parse_content_or_calls(current_text) or ""
+ curr_has_content_tag = self._has_content_tag(current_text)
+ curr_has_tool_calls_tag = self._has_tool_calls_tag(current_text)
+ curr_has_content_phase = curr_has_content_tag or curr_has_tool_calls_tag
+
+ # If content phase just appeared (either <|content|> or <|tool_calls|>),
+ # emit an empty content delta to initialize the content field in
+ # reconstructor even if no text yet. We never emit the tag itself as
+ # content. After that, we only emit content additions.
+ if curr_has_content_phase and not prev_has_content_phase:
+ return DeltaMessage(content="")
+
+ # If we have started content phase, we should emit only content deltas
+ if curr_has_content_phase:
+ if curr_c != prev_c:
+ addition = curr_c[len(prev_c):] if curr_c.startswith(prev_c) else curr_c
+ if addition:
+ return DeltaMessage(content=addition)
+ return None
+
+ # If neither reasoning nor content/tool_calls phases have started yet,
+ # emit raw delta as content immediately (e.g., "{" for JSON outputs).
+ if (
+ "<|think|>" not in current_text
+ and not self._has_content_phase(current_text)
+ and delta_text not in ("<|think|>", "<|end|>", "<|content|>", "<|tool_calls|>")
+ ):
+ return DeltaMessage(content=delta_text)
+
+ # Otherwise, emit reasoning progression between <|think|> and the first
+ # boundary (<|end|>, <|content|>, <|tool_calls|>). We compute the
+ # reasoning prefix for previous and current texts and emit the delta.
+ prev_prefix = self._parse_reasoning_prefix(previous_text) or ""
+ curr_prefix = self._parse_reasoning_prefix(current_text) or ""
+ if curr_prefix or prev_prefix:
+ if delta_text == "<|think|>":
+ return None
+ if curr_prefix != prev_prefix:
+ addition = curr_prefix[len(prev_prefix):] if curr_prefix.startswith(prev_prefix) else curr_prefix
+ if addition:
+ return DeltaMessage(reasoning=addition)
+
+ # Fallback: if we're clearly within reasoning (think seen, no boundary
+ # reached yet) and the delta is not a boundary token, emit it as
+ # reasoning. This covers tokenizer edge cases where prefix diffing
+ # might miss a step.
+ if (
+ ("<|think|>" in current_text)
+ and ("<|end|>" not in current_text)
+ and (not self._has_content_phase(current_text))
+ and delta_text not in ("<|think|>", "<|end|>", "<|content|>", "<|tool_calls|>")
+ ):
+ return DeltaMessage(reasoning=delta_text)
+
+ # Final guard: if we've already seen <|think|> in the previous_text and
+ # haven't started content/tool_calls or ended reasoning yet, emit any
+ # non-boundary delta as reasoning.
+ if (
+ ("<|think|>" in previous_text)
+ and ("<|end|>" not in previous_text)
+ and (not self._has_content_phase(previous_text))
+ and delta_text not in ("<|think|>", "<|end|>", "<|content|>", "<|tool_calls|>")
+ ):
+ return DeltaMessage(reasoning=delta_text)
+
+ return None
+
+ # --------------------
+ # Internal helpers
+ # --------------------
+ def _token_ids(self, text: str) -> list[int]:
+ tokenizer = self.model_tokenizer
+ tokens = tokenizer.tokenize(text)
+ return tokenizer.convert_tokens_to_ids(tokens)
+
+ def _find_subsequence(self, haystack: Sequence[int], needle: Sequence[int]) -> int:
+ if not needle:
+ return -1
+ n = len(needle)
+ limit = len(haystack) - n + 1
+ for i in range(limit):
+ if haystack[i:i + n] == list(needle):
+ return i
+ return -1
+
+ def _rfind_subsequence(self, haystack: Sequence[int], needle: Sequence[int]) -> int:
+ if not needle:
+ return -1
+ n = len(needle)
+ limit = len(haystack) - n
+ last = -1
+ for i in range(0, limit + 1):
+ if haystack[i:i + n] == list(needle):
+ last = i
+ return last
+
+ def _parse_reasoning(self, text: str) -> Optional[str]:
+ # Extract text between first <|think|> and subsequent <|end|>
+ think_tag = "<|think|>"
+ end_tag = "<|end|>"
+ s = text.find(think_tag)
+ if s == -1:
+ return None
+ s += len(think_tag)
+ e = text.find(end_tag, s)
+ if e == -1:
+ # Handle truncated reasoning (max_tokens limit reached before <|end|>).
+ # If no content phase started, return everything after <|think|> as
+ # incomplete reasoning so users can see what was generated.
+ if not self._has_content_phase(text[s:]):
+ return text[s:] if s < len(text) else None
+ return None
+ return text[s:e]
+
+ def _parse_trailing_content(self, text: str) -> Optional[str]:
+ # Return everything after the first <|content|> tag (including any trailing special tokens)
+ content_tag = "<|content|>"
+ s = text.find(content_tag)
+ if s == -1:
+ return None
+ s += len(content_tag)
+ if s >= len(text):
+ # Content tag exists but no trailing text -> empty content
+ return ""
+ return text[s:]
+
+ def _has_content_tag(self, text: str) -> bool:
+ return text.find("<|content|>") != -1
+
+ # New helpers covering both content and tool-calls phases
+ def _parse_content_or_calls(self, text: str) -> Optional[str]:
+ content_tag = "<|content|>"
+ tool_calls_tag = "<|tool_calls|>"
+
+ ci = text.find(content_tag)
+ ti = text.find(tool_calls_tag)
+
+ if ci != -1:
+ # everything after content tag
+ start = ci + len(content_tag)
+ return text[start:] if start <= len(text) else ""
+ if ti != -1:
+ # everything after tool_calls tag (exclusive)
+ start = ti + len(tool_calls_tag)
+ return text[start:] if start <= len(text) else ""
+ return None
+
+ def _has_tool_calls_tag(self, text: str) -> bool:
+ return text.find("<|tool_calls|>") != -1
+
+ def _has_content_phase(self, text: str) -> bool:
+ return self._has_content_tag(text) or self._has_tool_calls_tag(text)
+
+ def _is_in_reasoning_phase_prev(self, text: str) -> bool:
+ # Determine reasoning phase using the PREVIOUS text so that if the
+ # current delta includes boundary tokens merged with other text, we
+ # still emit the delta as reasoning unless the delta itself is a
+ # boundary token. This matches the test expectations.
+ if text.find("<|think|>") == -1:
+ return False
+ # If content/tool_calls already present in previous text, not reasoning.
+ if self._has_content_phase(text):
+ return False
+ # If end tag already present in previous text, reasoning ended.
+ if text.find("<|end|>") != -1:
+ return False
+ return True
+
+ def _starts_reasoning_now(self, text: str) -> bool:
+ # Returns True if current_text includes <|think|> but no boundary
+ # tokens after it yet. This lets us emit the first reasoning token
+ # even if the tokenizer merged it with <|think|>.
+ i = text.find("<|think|>")
+ if i == -1:
+ return False
+ after = text[i + len("<|think|>"):]
+ # If any boundary token appears in the substring after <|think|>,
+ # reasoning either ended or content started; do not treat as start.
+ for b in ("<|end|>", "<|content|>", "<|tool_calls|>"):
+ if after.find(b) != -1:
+ return False
+ return True
+
+ def _parse_reasoning_prefix(self, text: str) -> Optional[str]:
+ # Returns text between the first <|think|> and the earliest boundary
+ # among <|end|>, <|content|>, <|tool_calls|>. If <|think|> is absent,
+ # returns None. If no boundary appears, returns text after <|think|>.
+ ti = text.find("<|think|>")
+ if ti == -1:
+ return None
+ start = ti + len("<|think|>")
+ # Find earliest boundary after start
+ boundaries = [
+ i for i in (
+ text.find("<|end|>", start),
+ text.find("<|content|>", start),
+ text.find("<|tool_calls|>", start),
+ ) if i != -1
+ ]
+ end = min(boundaries) if boundaries else len(text)
+ return text[start:end]
diff --git a/solar_open_tool_parser.py b/solar_open_tool_parser.py
new file mode 100644
index 0000000000000000000000000000000000000000..0636e51424d9caa1e319e4fc53e8e23ec4e971f1
--- /dev/null
+++ b/solar_open_tool_parser.py
@@ -0,0 +1,267 @@
+# coding=utf-8
+# Copyright 2025 Upstage AI.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import random
+import re
+import string
+import ast
+import json
+from collections.abc import Sequence
+from typing import Union, Tuple, List, Optional
+
+from vllm.entrypoints.openai.protocol import (
+ ChatCompletionRequest,
+ DeltaMessage,
+ DeltaFunctionCall,
+ DeltaToolCall,
+ ExtractedToolCallInformation,
+ ToolCall,
+ FunctionCall,
+)
+from vllm.entrypoints.openai.tool_parsers.abstract_tool_parser import (
+ ToolParser
+)
+from vllm.logger import init_logger
+
+import pyjson5
+
+class ToolCallID:
+ _LENGTH = 10
+
+ def __init__(self, id_val: str, validation: bool = False):
+ self._id = id_val
+ if validation:
+ self._validate()
+
+ @classmethod
+ def random(cls, validation=False) -> 'ToolCallID':
+ chars = string.ascii_lowercase + string.digits
+ return cls(''.join(random.choice(chars) for _ in range(ToolCallID._LENGTH)), validation=validation)
+
+ def _validate(self):
+ assert len(self._id) == ToolCallID._LENGTH
+ pattern = r'^[a-z0-9]{10}$'
+ assert re.match(pattern, self._id) is not None
+
+ def to_string(self) -> str:
+ return self._id
+
+ def __str__(self) -> str:
+ return self.to_string()
+
+
+logger = init_logger(__name__)
+
+
+class SolarOpenToolParser(ToolParser):
+
+ def extract_tool_calls(
+ self,
+ model_output: str,
+ request: ChatCompletionRequest,
+ ) -> ExtractedToolCallInformation:
+ content, tool_calls = self._parse_text(model_output)
+ return ExtractedToolCallInformation(
+ tools_called=len(tool_calls) > 0,
+ tool_calls=tool_calls,
+ content=content if content else None,
+ )
+
+ def extract_tool_calls_streaming(
+ self,
+ previous_text: str,
+ current_text: str,
+ delta_text: str,
+ previous_token_ids: Sequence[int],
+ current_token_ids: Sequence[int],
+ delta_token_ids: Sequence[int],
+ request: ChatCompletionRequest,
+ ) -> Union[DeltaMessage, None]:
+ # 1) Emit plain content tokens immediately until content terminator
+ # tags or tool_calls section begins. Be careful when tokenizer groups
+ # multiple special tags into a single delta (e.g., "<|tool_calls|><|tool_call:begin|>").
+ # Only emit as content if BOTH:
+ # - previous_text has not seen any special markers, and
+ # - delta_text does NOT contain any of those markers as a substring.
+ if delta_text:
+ # Do NOT emit content if we have already started any special section
+ # including tool call tags. Content should only be emitted at the
+ # very beginning before any markers show up.
+ special_markers = (
+ "<|flush|>",
+ "<|end|>",
+ "<|begin|>",
+ "<|tool_calls|>",
+ "<|tool_call:begin|>",
+ "<|tool_call:name|>",
+ "<|tool_call:args|>",
+ "<|tool_call:end|>",
+ "<|calls|>",
+ )
+ if not any(tag in previous_text for tag in special_markers):
+ if not any(tag in delta_text for tag in special_markers):
+ return DeltaMessage(content=delta_text, tool_calls=[])
+
+ tool_call_deltas: list[DeltaToolCall] = []
+
+ # Helper lambdas to analyze current_text state
+ def _completed_calls_count(txt: str) -> int:
+ return len(self._parse_tool_calls(txt))
+
+ # Detect if a new tool_call started streaming its args just now.
+ if delta_text and "<|tool_call:args|>" in delta_text:
+ # Extract id and name for the latest tool call block present so far.
+ begin_tag = "<|tool_call:begin|>"
+ name_tag = "<|tool_call:name|>"
+ args_tag = "<|tool_call:args|>"
+
+ latest_args = current_text.rfind(args_tag)
+ latest_name = current_text.rfind(name_tag, 0, latest_args if latest_args != -1 else None)
+ latest_begin = current_text.rfind(begin_tag, 0, latest_name if latest_name != -1 else None)
+ if latest_begin != -1 and latest_name != -1 and latest_args != -1 and latest_begin < latest_name < latest_args:
+ tool_id = current_text[latest_begin + len(begin_tag):latest_name]
+ func_name = current_text[latest_name + len(name_tag):latest_args]
+ # Index equals number of args tags seen before this delta
+ index = previous_text.count(args_tag)
+ tool_call_deltas.append(
+ DeltaToolCall(
+ id=tool_id,
+ type="function",
+ index=index,
+ function=DeltaFunctionCall(name=func_name, arguments=""),
+ )
+ )
+
+ # If we are inside args (after last args tag without end), stream arg chunk
+ begin_tag = "<|tool_call:begin|>"
+ args_tag = "<|tool_call:args|>"
+ end_tag = "<|tool_call:end|>"
+ last_args_pos = current_text.rfind(args_tag)
+ last_end_pos = current_text.rfind(end_tag)
+ if last_args_pos != -1 and (last_end_pos == -1 or last_args_pos > last_end_pos):
+ # Currently within args for the latest tool call
+ # Determine previous args text and current args text to compute delta
+ prev_last_args = previous_text.rfind(args_tag)
+ prev_last_end = previous_text.rfind(end_tag)
+ if prev_last_args != -1 and (prev_last_end == -1 or prev_last_args > prev_last_end):
+ # Already inside args previously: emit only the delta_text
+ if delta_text and delta_text not in (begin_tag, args_tag, end_tag):
+ # Stream into the most recently started (but not yet ended) call
+ index = max(previous_text.count(args_tag) - 1, 0)
+ tool_call_deltas.append(
+ DeltaToolCall(
+ id=None,
+ type=None,
+ index=index,
+ function=DeltaFunctionCall(name=None, arguments=delta_text),
+ )
+ )
+
+ if not tool_call_deltas:
+ return None
+
+ return DeltaMessage(content=None, tool_calls=tool_call_deltas)
+
+ # --------------------
+ # Internal helpers
+ # --------------------
+ def _parse_text(self, text: str) -> Tuple[Optional[str], List[ToolCall]]:
+ """Parse the completed segments from the given text.
+
+ Returns (content, tool_calls) where content is extracted as the leading
+ text up to the first '<|flush|>' or '<|end|>' marker, and tool_calls is
+ a list of fully parsed tool calls inside '<|tool_calls|> ... <|calls|>'.
+ """
+ content = self._parse_content(text)
+ tool_calls = self._parse_tool_calls(text)
+ return content, tool_calls
+
+ def _parse_content(self, text: str) -> Optional[str]:
+ """Extract assistant content from the text.
+
+ Rule: take the leading content before the first '<|flush|>' or
+ '<|end|>' marker. If neither marker exists, return None.
+ """
+ end_tags = ["<|flush|>", "<|end|>"]
+
+ # Take leading content before the first end tag
+ end_positions = [pos for tag in end_tags if (pos := text.find(tag)) != -1]
+ if not end_positions:
+ return None
+ end = min(end_positions)
+ # Trim only the extracted portion; tests expect exact substring
+ return text[:end]
+
+ def _parse_tool_call_args(self, text: str) -> str:
+ try:
+ # Try to parse as JSON
+ args = json.loads(text)
+ except json.JSONDecodeError:
+ try:
+ # Try to parse as JSON5
+ args = pyjson5.decode(text)
+ except pyjson5.Json5DecoderException:
+ try:
+ # Try to parse as Python literal
+ args = ast.literal_eval(text)
+ except Exception:
+ # Fallback: return the original string
+ args = text
+ if not isinstance(args, str):
+ # Always convert back to JSON string
+ args = json.dumps(args)
+ return args
+
+ def _parse_tool_calls(self, text: str) -> List[ToolCall]:
+ tool_calls: list[ToolCall] = []
+ # Parse globally; wrapper '<|tool_calls|>' may or may not be present.
+ section_start = 0
+ # section ends at <|calls|> if present, else use end of text
+ section_end = text.find("<|calls|>")
+ if section_end == -1:
+ section_end = len(text)
+ i = section_start
+ while True:
+ begin_tag = "<|tool_call:begin|>"
+ name_tag = "<|tool_call:name|>"
+ args_tag = "<|tool_call:args|>"
+ end_tag = "<|tool_call:end|>"
+
+ b = text.find(begin_tag, i, section_end)
+ if b == -1:
+ break
+ b += len(begin_tag)
+ n = text.find(name_tag, b, section_end)
+ if n == -1:
+ break
+ tool_id = text[b:n]
+ n += len(name_tag)
+ a = text.find(args_tag, n, section_end)
+ if a == -1:
+ break
+ name = text[n:a]
+ a += len(args_tag)
+ e = text.find(end_tag, a, section_end)
+ if e == -1:
+ break
+ args = text[a:e]
+ tool_calls.append(
+ ToolCall(
+ id=tool_id,
+ function=FunctionCall(name=name, arguments=self._parse_tool_call_args(args)),
+ ))
+ i = e + len(end_tag)
+
+ return tool_calls
diff --git a/special_tokens_map.json b/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..a506572e61096d1d324556cf55f38babc0bf14e2
--- /dev/null
+++ b/special_tokens_map.json
@@ -0,0 +1,4006 @@
+{
+ "additional_special_tokens": [
+ "",
+ "<|startoftext|>",
+ "<|endoftext|>",
+ "<|fim_prefix|>",
+ "<|fim_middle|>",
+ "<|fim_suffix|>",
+ "<|special_6|>",
+ "<|special_7|>",
+ "<|special_8|>",
+ "<|special_9|>",
+ "<|special_10|>",
+ "<|special_11|>",
+ "<|special_12|>",
+ "<|special_13|>",
+ "<|special_14|>",
+ "<|special_15|>",
+ "<|special_16|>",
+ "<|special_17|>",
+ "<|special_18|>",
+ "<|special_19|>",
+ "<|flush|>",
+ "<|calls|>",
+ "<|tools:begin|>",
+ "<|tools:end|>",
+ "<|tool:begin|>",
+ "<|tool:end|>",
+ "<|tool_response|>",
+ "<|tool_response:begin|>",
+ "<|tool_response:end|>",
+ "<|tool_response:name|>",
+ "<|tool_response:result|>",
+ "<|special_40|>",
+ "<|special_41|>",
+ "<|special_42|>",
+ "<|special_43|>",
+ "<|special_44|>",
+ "<|special_45|>",
+ "<|special_46|>",
+ "<|special_47|>",
+ "<|special_48|>",
+ "<|special_49|>",
+ "<|special_50|>",
+ "<|special_51|>",
+ "<|special_52|>",
+ "<|special_53|>",
+ "<|special_54|>",
+ "<|special_55|>",
+ "<|special_56|>",
+ "<|special_57|>",
+ "<|special_58|>",
+ "<|special_59|>",
+ "<|special_60|>",
+ "<|special_61|>",
+ "<|special_62|>",
+ "<|special_63|>",
+ "<|special_64|>",
+ "<|special_65|>",
+ "<|special_66|>",
+ "<|special_67|>",
+ "<|special_68|>",
+ "<|special_69|>",
+ "<|special_70|>",
+ "<|special_71|>",
+ "<|special_72|>",
+ "<|special_73|>",
+ "<|special_74|>",
+ "<|special_75|>",
+ "<|special_76|>",
+ "<|special_77|>",
+ "<|special_78|>",
+ "<|special_79|>",
+ "<|special_80|>",
+ "<|special_81|>",
+ "<|special_82|>",
+ "<|special_83|>",
+ "<|special_84|>",
+ "<|special_85|>",
+ "<|special_86|>",
+ "<|special_87|>",
+ "<|special_88|>",
+ "<|special_89|>",
+ "<|special_90|>",
+ "<|special_91|>",
+ "<|special_92|>",
+ "<|special_93|>",
+ "<|special_94|>",
+ "<|special_95|>",
+ "<|special_96|>",
+ "<|special_97|>",
+ "<|special_98|>",
+ "<|special_99|>",
+ "<|special_100|>",
+ "<|special_101|>",
+ "<|special_102|>",
+ "<|special_103|>",
+ "<|special_104|>",
+ "<|special_105|>",
+ "<|special_106|>",
+ "<|special_107|>",
+ "<|special_108|>",
+ "<|special_109|>",
+ "<|special_110|>",
+ "<|special_111|>",
+ "<|special_112|>",
+ "<|special_113|>",
+ "<|special_114|>",
+ "<|special_115|>",
+ "<|special_116|>",
+ "<|special_117|>",
+ "<|special_118|>",
+ "<|special_119|>",
+ "<|special_120|>",
+ "<|special_121|>",
+ "<|special_122|>",
+ "<|special_123|>",
+ "<|special_124|>",
+ "<|special_125|>",
+ "<|special_126|>",
+ "<|special_127|>",
+ "<|special_128|>",
+ "<|special_129|>",
+ "<|special_130|>",
+ "<|special_131|>",
+ "<|special_132|>",
+ "<|special_133|>",
+ "<|special_134|>",
+ "<|special_135|>",
+ "<|special_136|>",
+ "<|special_137|>",
+ "<|special_138|>",
+ "<|special_139|>",
+ "<|special_140|>",
+ "<|special_141|>",
+ "<|special_142|>",
+ "<|special_143|>",
+ "<|special_144|>",
+ "<|special_145|>",
+ "<|special_146|>",
+ "<|special_147|>",
+ "<|special_148|>",
+ "<|special_149|>",
+ "<|special_150|>",
+ "<|special_151|>",
+ "<|special_152|>",
+ "<|special_153|>",
+ "<|special_154|>",
+ "<|special_155|>",
+ "<|special_156|>",
+ "<|special_157|>",
+ "<|special_158|>",
+ "<|special_159|>",
+ "<|special_160|>",
+ "<|special_161|>",
+ "<|special_162|>",
+ "<|special_163|>",
+ "<|special_164|>",
+ "<|special_165|>",
+ "<|special_166|>",
+ "<|special_167|>",
+ "<|special_168|>",
+ "<|special_169|>",
+ "<|special_170|>",
+ "<|special_171|>",
+ "<|special_172|>",
+ "<|special_173|>",
+ "<|special_174|>",
+ "<|special_175|>",
+ "<|special_176|>",
+ "<|special_177|>",
+ "<|special_178|>",
+ "<|special_179|>",
+ "<|special_180|>",
+ "<|special_181|>",
+ "<|special_182|>",
+ "<|special_183|>",
+ "<|special_184|>",
+ "<|special_185|>",
+ "<|special_186|>",
+ "<|special_187|>",
+ "<|special_188|>",
+ "<|special_189|>",
+ "<|special_190|>",
+ "<|special_191|>",
+ "<|special_192|>",
+ "<|special_193|>",
+ "<|special_194|>",
+ "<|special_195|>",
+ "<|special_196|>",
+ "<|special_197|>",
+ "<|special_198|>",
+ "<|special_199|>",
+ "<|special_200|>",
+ "<|special_201|>",
+ "<|special_202|>",
+ "<|special_203|>",
+ "<|special_204|>",
+ "<|special_205|>",
+ "<|special_206|>",
+ "<|special_207|>",
+ "<|special_208|>",
+ "<|special_209|>",
+ "<|special_210|>",
+ "<|special_211|>",
+ "<|special_212|>",
+ "<|special_213|>",
+ "<|special_214|>",
+ "<|special_215|>",
+ "<|special_216|>",
+ "<|special_217|>",
+ "<|special_218|>",
+ "<|special_219|>",
+ "<|special_220|>",
+ "<|special_221|>",
+ "<|special_222|>",
+ "<|special_223|>",
+ "<|special_224|>",
+ "<|special_225|>",
+ "<|special_226|>",
+ "<|special_227|>",
+ "<|special_228|>",
+ "<|special_229|>",
+ "<|special_230|>",
+ "<|special_231|>",
+ "<|special_232|>",
+ "<|special_233|>",
+ "<|special_234|>",
+ "<|special_235|>",
+ "<|special_236|>",
+ "<|special_237|>",
+ "<|special_238|>",
+ "<|special_239|>",
+ "<|special_240|>",
+ "<|special_241|>",
+ "<|special_242|>",
+ "<|special_243|>",
+ "<|special_244|>",
+ "<|special_245|>",
+ "<|special_246|>",
+ "<|special_247|>",
+ "<|special_248|>",
+ "<|special_249|>",
+ "<|special_250|>",
+ "<|special_251|>",
+ "<|special_252|>",
+ "<|special_253|>",
+ "<|special_254|>",
+ "<|special_255|>",
+ "<|special_256|>",
+ "<|special_257|>",
+ "<|special_258|>",
+ "<|special_259|>",
+ "<|special_260|>",
+ "<|special_261|>",
+ "<|special_262|>",
+ "<|special_263|>",
+ "<|special_264|>",
+ "<|special_265|>",
+ "<|special_266|>",
+ "<|special_267|>",
+ "<|special_268|>",
+ "<|special_269|>",
+ "<|special_270|>",
+ "<|special_271|>",
+ "<|special_272|>",
+ "<|special_273|>",
+ "<|special_274|>",
+ "<|special_275|>",
+ "<|special_276|>",
+ "<|special_277|>",
+ "<|special_278|>",
+ "<|special_279|>",
+ "<|special_280|>",
+ "<|special_281|>",
+ "<|special_282|>",
+ "<|special_283|>",
+ "<|special_284|>",
+ "<|special_285|>",
+ "<|special_286|>",
+ "<|special_287|>",
+ "<|special_288|>",
+ "<|special_289|>",
+ "<|special_290|>",
+ "<|special_291|>",
+ "<|special_292|>",
+ "<|special_293|>",
+ "<|special_294|>",
+ "<|special_295|>",
+ "<|special_296|>",
+ "<|special_297|>",
+ "<|special_298|>",
+ "<|special_299|>",
+ "<|special_300|>",
+ "<|special_301|>",
+ "<|special_302|>",
+ "<|special_303|>",
+ "<|special_304|>",
+ "<|special_305|>",
+ "<|special_306|>",
+ "<|special_307|>",
+ "<|special_308|>",
+ "<|special_309|>",
+ "<|special_310|>",
+ "<|special_311|>",
+ "<|special_312|>",
+ "<|special_313|>",
+ "<|special_314|>",
+ "<|special_315|>",
+ "<|special_316|>",
+ "<|special_317|>",
+ "<|special_318|>",
+ "<|special_319|>",
+ "<|special_320|>",
+ "<|special_321|>",
+ "<|special_322|>",
+ "<|special_323|>",
+ "<|special_324|>",
+ "<|special_325|>",
+ "<|special_326|>",
+ "<|special_327|>",
+ "<|special_328|>",
+ "<|special_329|>",
+ "<|special_330|>",
+ "<|special_331|>",
+ "<|special_332|>",
+ "<|special_333|>",
+ "<|special_334|>",
+ "<|special_335|>",
+ "<|special_336|>",
+ "<|special_337|>",
+ "<|special_338|>",
+ "<|special_339|>",
+ "<|special_340|>",
+ "<|special_341|>",
+ "<|special_342|>",
+ "<|special_343|>",
+ "<|special_344|>",
+ "<|special_345|>",
+ "<|special_346|>",
+ "<|special_347|>",
+ "<|special_348|>",
+ "<|special_349|>",
+ "<|special_350|>",
+ "<|special_351|>",
+ "<|special_352|>",
+ "<|special_353|>",
+ "<|special_354|>",
+ "<|special_355|>",
+ "<|special_356|>",
+ "<|special_357|>",
+ "<|special_358|>",
+ "<|special_359|>",
+ "<|special_360|>",
+ "<|special_361|>",
+ "<|special_362|>",
+ "<|special_363|>",
+ "<|special_364|>",
+ "<|special_365|>",
+ "<|special_366|>",
+ "<|special_367|>",
+ "<|special_368|>",
+ "<|special_369|>",
+ "<|special_370|>",
+ "<|special_371|>",
+ "<|special_372|>",
+ "<|special_373|>",
+ "<|special_374|>",
+ "<|special_375|>",
+ "<|special_376|>",
+ "<|special_377|>",
+ "<|special_378|>",
+ "<|special_379|>",
+ "<|special_380|>",
+ "<|special_381|>",
+ "<|special_382|>",
+ "<|special_383|>",
+ "<|special_384|>",
+ "<|special_385|>",
+ "<|special_386|>",
+ "<|special_387|>",
+ "<|special_388|>",
+ "<|special_389|>",
+ "<|special_390|>",
+ "<|special_391|>",
+ "<|special_392|>",
+ "<|special_393|>",
+ "<|special_394|>",
+ "<|special_395|>",
+ "<|special_396|>",
+ "<|special_397|>",
+ "<|special_398|>",
+ "<|special_399|>",
+ "<|special_400|>",
+ "<|special_401|>",
+ "<|special_402|>",
+ "<|special_403|>",
+ "<|special_404|>",
+ "<|special_405|>",
+ "<|special_406|>",
+ "<|special_407|>",
+ "<|special_408|>",
+ "<|special_409|>",
+ "<|special_410|>",
+ "<|special_411|>",
+ "<|special_412|>",
+ "<|special_413|>",
+ "<|special_414|>",
+ "<|special_415|>",
+ "<|special_416|>",
+ "<|special_417|>",
+ "<|special_418|>",
+ "<|special_419|>",
+ "<|special_420|>",
+ "<|special_421|>",
+ "<|special_422|>",
+ "<|special_423|>",
+ "<|special_424|>",
+ "<|special_425|>",
+ "<|special_426|>",
+ "<|special_427|>",
+ "<|special_428|>",
+ "<|special_429|>",
+ "<|special_430|>",
+ "<|special_431|>",
+ "<|special_432|>",
+ "<|special_433|>",
+ "<|special_434|>",
+ "<|special_435|>",
+ "<|special_436|>",
+ "<|special_437|>",
+ "<|special_438|>",
+ "<|special_439|>",
+ "<|special_440|>",
+ "<|special_441|>",
+ "<|special_442|>",
+ "<|special_443|>",
+ "<|special_444|>",
+ "<|special_445|>",
+ "<|special_446|>",
+ "<|special_447|>",
+ "<|special_448|>",
+ "<|special_449|>",
+ "<|special_450|>",
+ "<|special_451|>",
+ "<|special_452|>",
+ "<|special_453|>",
+ "<|special_454|>",
+ "<|special_455|>",
+ "<|special_456|>",
+ "<|special_457|>",
+ "<|special_458|>",
+ "<|special_459|>",
+ "<|special_460|>",
+ "<|special_461|>",
+ "<|special_462|>",
+ "<|special_463|>",
+ "<|special_464|>",
+ "<|special_465|>",
+ "<|special_466|>",
+ "<|special_467|>",
+ "<|special_468|>",
+ "<|special_469|>",
+ "<|special_470|>",
+ "<|special_471|>",
+ "<|special_472|>",
+ "<|special_473|>",
+ "<|special_474|>",
+ "<|special_475|>",
+ "<|special_476|>",
+ "<|special_477|>",
+ "<|special_478|>",
+ "<|special_479|>",
+ "<|special_480|>",
+ "<|special_481|>",
+ "<|special_482|>",
+ "<|special_483|>",
+ "<|special_484|>",
+ "<|special_485|>",
+ "<|special_486|>",
+ "<|special_487|>",
+ "<|special_488|>",
+ "<|special_489|>",
+ "<|special_490|>",
+ "<|special_491|>",
+ "<|special_492|>",
+ "<|special_493|>",
+ "<|special_494|>",
+ "<|special_495|>",
+ "<|special_496|>",
+ "<|special_497|>",
+ "<|special_498|>",
+ "<|special_499|>",
+ "<|special_500|>",
+ "<|special_501|>",
+ "<|special_502|>",
+ "<|special_503|>",
+ "<|special_504|>",
+ "<|special_505|>",
+ "<|special_506|>",
+ "<|special_507|>",
+ "<|special_508|>",
+ "<|special_509|>",
+ "<|special_510|>",
+ "<|special_511|>",
+ "<|special_625|>",
+ "<|special_626|>",
+ "<|special_627|>",
+ "<|special_628|>",
+ "<|special_629|>",
+ "<|special_630|>",
+ "<|special_631|>",
+ "<|special_632|>",
+ "<|special_633|>",
+ "<|special_634|>",
+ "<|special_635|>",
+ "<|special_636|>",
+ "<|special_637|>",
+ "<|special_638|>",
+ "<|special_639|>",
+ "<|special_640|>",
+ "<|special_641|>",
+ "<|special_642|>",
+ "<|special_643|>",
+ "<|special_644|>",
+ "<|special_645|>",
+ "<|special_646|>",
+ "<|special_647|>",
+ "<|special_648|>",
+ "<|special_649|>",
+ "<|special_650|>",
+ "<|special_651|>",
+ "<|special_652|>",
+ "<|special_653|>",
+ "<|special_654|>",
+ "<|special_655|>",
+ "<|special_656|>",
+ "<|special_657|>",
+ "<|special_658|>",
+ "<|special_659|>",
+ "<|special_660|>",
+ "<|special_661|>",
+ "<|special_662|>",
+ "<|special_663|>",
+ "<|special_664|>",
+ "<|special_665|>",
+ "<|special_666|>",
+ "<|special_667|>",
+ "<|special_668|>",
+ "<|special_669|>",
+ "<|special_670|>",
+ "<|special_671|>",
+ "<|special_672|>",
+ "<|special_673|>",
+ "<|special_674|>",
+ "<|special_675|>",
+ "<|special_676|>",
+ "<|special_677|>",
+ "<|special_678|>",
+ "<|special_679|>",
+ "<|special_680|>",
+ "<|special_681|>",
+ "<|special_682|>",
+ "<|special_683|>",
+ "<|special_684|>",
+ "<|special_685|>",
+ "<|special_686|>",
+ "<|special_687|>",
+ "<|special_688|>",
+ "<|special_689|>",
+ "<|special_690|>",
+ "<|special_691|>",
+ "<|special_692|>",
+ "<|special_693|>",
+ "<|special_694|>",
+ "<|special_695|>",
+ "<|special_696|>",
+ "<|special_697|>",
+ "<|special_698|>",
+ "<|special_699|>",
+ "<|special_700|>",
+ "<|special_701|>",
+ "<|special_702|>",
+ "<|special_703|>",
+ "<|special_704|>",
+ "<|special_705|>",
+ "<|special_706|>",
+ "<|special_707|>",
+ "<|special_708|>",
+ "<|special_709|>",
+ "<|special_710|>",
+ "<|special_711|>",
+ "<|special_712|>",
+ "<|special_713|>",
+ "<|special_714|>",
+ "<|special_715|>",
+ "<|special_716|>",
+ "<|special_717|>",
+ "<|special_718|>",
+ "<|special_719|>",
+ "<|special_720|>",
+ "<|special_721|>",
+ "<|special_722|>",
+ "<|special_723|>",
+ "<|special_724|>",
+ "<|special_725|>",
+ "<|special_726|>",
+ "<|special_727|>",
+ "<|special_728|>",
+ "<|special_729|>",
+ "<|special_730|>",
+ "<|special_731|>",
+ "<|special_732|>",
+ "<|special_733|>",
+ "<|special_734|>",
+ "<|special_735|>",
+ "<|special_736|>",
+ "<|special_737|>",
+ "<|special_738|>",
+ "<|special_739|>",
+ "<|special_740|>",
+ "<|special_741|>",
+ "<|special_742|>",
+ "<|special_743|>",
+ "<|special_744|>",
+ "<|special_745|>",
+ "<|special_746|>",
+ "<|special_747|>",
+ "<|special_748|>",
+ "<|special_749|>",
+ "<|special_750|>",
+ "<|special_751|>",
+ "<|special_752|>",
+ "<|special_753|>",
+ "<|special_754|>",
+ "<|special_755|>",
+ "<|special_756|>",
+ "<|special_757|>",
+ "<|special_758|>",
+ "<|special_759|>",
+ "<|special_760|>",
+ "<|special_761|>",
+ "<|special_762|>",
+ "<|special_763|>",
+ "<|special_764|>",
+ "<|special_765|>",
+ "<|special_766|>",
+ "<|special_767|>",
+ "<|special_768|>",
+ "<|special_769|>",
+ "<|special_770|>",
+ "<|special_771|>",
+ "<|special_772|>",
+ "<|special_773|>",
+ "<|special_774|>",
+ "<|special_775|>",
+ "<|special_776|>",
+ "<|special_777|>",
+ "<|special_778|>",
+ "<|special_779|>",
+ "<|special_780|>",
+ "<|special_781|>",
+ "<|special_782|>",
+ "<|special_783|>",
+ "<|special_784|>",
+ "<|special_785|>",
+ "<|special_786|>",
+ "<|special_787|>",
+ "<|special_788|>",
+ "<|special_789|>",
+ "<|special_790|>",
+ "<|special_791|>",
+ "<|special_792|>",
+ "<|special_793|>",
+ "<|special_794|>",
+ "<|special_795|>",
+ "<|special_796|>",
+ "<|special_797|>",
+ "<|special_798|>",
+ "<|special_799|>",
+ "<|special_800|>",
+ "<|special_801|>",
+ "<|special_802|>",
+ "<|special_803|>",
+ "<|special_804|>",
+ "<|special_805|>",
+ "<|special_806|>",
+ "<|special_807|>",
+ "<|special_808|>",
+ "<|special_809|>",
+ "<|special_810|>",
+ "<|special_811|>",
+ "<|special_812|>",
+ "<|special_813|>",
+ "<|special_814|>",
+ "<|special_815|>",
+ "<|special_816|>",
+ "<|special_817|>",
+ "<|special_818|>",
+ "<|special_819|>",
+ "<|special_820|>",
+ "<|special_821|>",
+ "<|special_822|>",
+ "<|special_823|>",
+ "<|special_824|>",
+ "<|special_825|>",
+ "<|special_826|>",
+ "<|special_827|>",
+ "<|special_828|>",
+ "<|special_829|>",
+ "<|special_830|>",
+ "<|special_831|>",
+ "<|special_832|>",
+ "<|special_833|>",
+ "<|special_834|>",
+ "<|special_835|>",
+ "<|special_836|>",
+ "<|special_837|>",
+ "<|special_838|>",
+ "<|special_839|>",
+ "<|special_840|>",
+ "<|special_841|>",
+ "<|special_842|>",
+ "<|special_843|>",
+ "<|special_844|>",
+ "<|special_845|>",
+ "<|special_846|>",
+ "<|special_847|>",
+ "<|special_848|>",
+ "<|special_849|>",
+ "<|special_850|>",
+ "<|special_851|>",
+ "<|special_852|>",
+ "<|special_853|>",
+ "<|special_854|>",
+ "<|special_855|>",
+ "<|special_856|>",
+ "<|special_857|>",
+ "<|special_858|>",
+ "<|special_859|>",
+ "<|special_860|>",
+ "<|special_861|>",
+ "<|special_862|>",
+ "<|special_863|>",
+ "<|special_864|>",
+ "<|special_865|>",
+ "<|special_866|>",
+ "<|special_867|>",
+ "<|special_868|>",
+ "<|special_869|>",
+ "<|special_870|>",
+ "<|special_871|>",
+ "<|special_872|>",
+ "<|special_873|>",
+ "<|special_874|>",
+ "<|special_875|>",
+ "<|special_876|>",
+ "<|special_877|>",
+ "<|special_878|>",
+ "<|special_879|>",
+ "<|special_880|>",
+ "<|special_881|>",
+ "<|special_882|>",
+ "<|special_883|>",
+ "<|special_884|>",
+ "<|special_885|>",
+ "<|special_886|>",
+ "<|special_887|>",
+ "<|special_888|>",
+ "<|special_889|>",
+ "<|special_890|>",
+ "<|special_891|>",
+ "<|special_892|>",
+ "<|special_893|>",
+ "<|special_894|>",
+ "<|special_895|>",
+ "<|special_896|>",
+ "<|special_897|>",
+ "<|special_898|>",
+ "<|special_899|>",
+ "<|special_900|>",
+ "<|special_901|>",
+ "<|special_902|>",
+ "<|special_903|>",
+ "<|special_904|>",
+ "<|special_905|>",
+ "<|special_906|>",
+ "<|special_907|>",
+ "<|special_908|>",
+ "<|special_909|>",
+ "<|special_910|>",
+ "<|special_911|>",
+ "<|special_912|>",
+ "<|special_913|>",
+ "<|special_914|>",
+ "<|special_915|>",
+ "<|special_916|>",
+ "<|special_917|>",
+ "<|special_918|>",
+ "<|special_919|>",
+ "<|special_920|>",
+ "<|special_921|>",
+ "<|special_922|>",
+ "<|special_923|>",
+ "<|special_924|>",
+ "<|special_925|>",
+ "<|special_926|>",
+ "<|special_927|>",
+ "<|special_928|>",
+ "<|special_929|>",
+ "<|special_930|>",
+ "<|special_931|>",
+ "<|special_932|>",
+ "<|special_933|>",
+ "<|special_934|>",
+ "<|special_935|>",
+ "<|special_936|>",
+ "<|special_937|>",
+ "<|special_938|>",
+ "<|special_939|>",
+ "<|special_940|>",
+ "<|special_941|>",
+ "<|special_942|>",
+ "<|special_943|>",
+ "<|special_944|>",
+ "<|special_945|>",
+ "<|special_946|>",
+ "<|special_947|>",
+ "<|special_948|>",
+ "<|special_949|>",
+ "<|special_950|>",
+ "<|special_951|>",
+ "<|special_952|>",
+ "<|special_953|>",
+ "<|special_954|>",
+ "<|special_955|>",
+ "<|special_956|>",
+ "<|special_957|>",
+ "<|special_958|>",
+ "<|special_959|>",
+ "<|special_960|>",
+ "<|special_961|>",
+ "<|special_962|>",
+ "<|special_963|>",
+ "<|special_964|>",
+ "<|special_965|>",
+ "<|special_966|>",
+ "<|special_967|>",
+ "<|special_968|>",
+ "<|special_969|>",
+ "<|special_970|>",
+ "<|special_971|>",
+ "<|special_972|>",
+ "<|special_973|>",
+ "<|special_974|>",
+ "<|special_975|>",
+ "<|special_976|>",
+ "<|special_977|>",
+ "<|special_978|>",
+ "<|special_979|>",
+ "<|special_980|>",
+ "<|special_981|>",
+ "<|special_982|>",
+ "<|special_983|>",
+ "<|special_984|>",
+ "<|special_985|>",
+ "<|special_986|>",
+ "<|special_987|>",
+ "<|special_988|>",
+ "<|special_989|>",
+ "<|special_990|>",
+ "<|special_991|>",
+ "<|special_992|>",
+ "<|special_993|>",
+ "<|special_994|>",
+ "<|special_995|>",
+ "<|special_996|>",
+ "<|special_997|>",
+ "<|special_998|>",
+ "<|special_999|>",
+ "<|special_1000|>",
+ "<|special_1001|>",
+ "<|special_1002|>",
+ "<|special_1003|>",
+ "<|special_1004|>",
+ "<|special_1005|>",
+ "<|special_1006|>",
+ "<|special_1007|>",
+ "<|special_1008|>",
+ "<|special_1009|>",
+ "<|special_1010|>",
+ "<|special_1011|>",
+ "<|special_1012|>",
+ "<|special_1013|>",
+ "<|special_1014|>",
+ "<|special_1015|>",
+ "<|special_1016|>",
+ "<|special_1017|>",
+ "<|special_1018|>",
+ "<|special_1019|>",
+ "<|special_1020|>",
+ "<|special_1021|>",
+ "<|special_1022|>",
+ "<|special_1023|>",
+ "<|special_1024|>",
+ "<|special_1025|>",
+ "<|special_1026|>",
+ "<|special_1027|>",
+ "<|special_1028|>",
+ "<|special_1029|>",
+ "<|special_1030|>",
+ "<|special_1031|>",
+ "<|special_1032|>",
+ "<|special_1033|>",
+ "<|special_1034|>",
+ "<|special_1035|>",
+ "<|special_1036|>",
+ "<|special_1037|>",
+ "<|special_1038|>",
+ "<|special_1039|>",
+ "<|special_1040|>",
+ "<|special_1041|>",
+ "<|special_1042|>",
+ "<|special_1043|>",
+ "<|special_1044|>",
+ "<|special_1045|>",
+ "<|special_1046|>",
+ "<|special_1047|>",
+ "<|special_1048|>",
+ "<|special_1049|>",
+ "<|special_1050|>",
+ "<|special_1051|>",
+ "<|special_1052|>",
+ "<|special_1053|>",
+ "<|special_1054|>",
+ "<|special_1055|>",
+ "<|special_1056|>",
+ "<|special_1057|>",
+ "<|special_1058|>",
+ "<|special_1059|>",
+ "<|special_1060|>",
+ "<|special_1061|>",
+ "<|special_1062|>",
+ "<|special_1063|>",
+ "<|special_1064|>",
+ "<|special_1065|>",
+ "<|special_1066|>",
+ "<|special_1067|>",
+ "<|special_1068|>",
+ "<|special_1069|>",
+ "<|special_1070|>",
+ "<|special_1071|>",
+ "<|special_1072|>",
+ "<|special_1073|>",
+ "<|special_1074|>",
+ "<|special_1075|>",
+ "<|special_1076|>",
+ "<|special_1077|>",
+ "<|special_1078|>",
+ "<|special_1079|>",
+ "<|special_1080|>",
+ "<|special_1081|>",
+ "<|special_1082|>",
+ "<|special_1083|>",
+ "<|special_1084|>",
+ "<|special_1085|>",
+ "<|special_1086|>",
+ "<|special_1087|>",
+ "<|special_1088|>",
+ "<|special_1089|>",
+ "<|special_1090|>",
+ "<|special_1091|>",
+ "<|special_1092|>",
+ "<|special_1093|>",
+ "<|special_1094|>",
+ "<|special_1095|>",
+ "<|special_1096|>",
+ "<|special_1097|>",
+ "<|special_1098|>",
+ "<|special_1099|>",
+ "<|special_1100|>",
+ "<|special_1101|>",
+ "<|special_1102|>",
+ "<|special_1103|>",
+ "<|special_1104|>",
+ "<|special_1105|>",
+ "<|special_1106|>",
+ "<|special_1107|>",
+ "<|special_1108|>",
+ "<|special_1109|>",
+ "<|special_1110|>",
+ "<|special_1111|>",
+ "<|special_1112|>",
+ "<|special_1113|>",
+ "<|special_1114|>",
+ "<|special_1115|>",
+ "<|special_1116|>",
+ "<|special_1117|>",
+ "<|special_1118|>",
+ "<|special_1119|>",
+ "<|special_1120|>",
+ "<|special_1121|>",
+ "<|special_1122|>",
+ "<|special_1123|>",
+ "<|special_1124|>",
+ "<|special_1125|>",
+ "<|special_1126|>",
+ "<|special_1127|>",
+ "<|special_1128|>",
+ "<|special_1129|>",
+ "<|special_1130|>",
+ "<|special_1131|>",
+ "<|special_1132|>",
+ "<|special_1133|>",
+ "<|special_1134|>",
+ "<|special_1135|>",
+ "<|special_1136|>",
+ "<|special_1137|>",
+ "<|special_1138|>",
+ "<|special_1139|>",
+ "<|special_1140|>",
+ "<|special_1141|>",
+ "<|special_1142|>",
+ "<|special_1143|>",
+ "<|special_1144|>",
+ "<|special_1145|>",
+ "<|special_1146|>",
+ "<|special_1147|>",
+ "<|special_1148|>",
+ "<|special_1149|>",
+ "<|special_1150|>",
+ "<|special_1151|>",
+ "<|special_1152|>",
+ "<|special_1153|>",
+ "<|special_1154|>",
+ "<|special_1155|>",
+ "<|special_1156|>",
+ "<|special_1157|>",
+ "<|special_1158|>",
+ "<|special_1159|>",
+ "<|special_1160|>",
+ "<|special_1161|>",
+ "<|special_1162|>",
+ "<|special_1163|>",
+ "<|special_1164|>",
+ "<|special_1165|>",
+ "<|special_1166|>",
+ "<|special_1167|>",
+ "<|special_1168|>",
+ "<|special_1169|>",
+ "<|special_1170|>",
+ "<|special_1171|>",
+ "<|special_1172|>",
+ "<|special_1173|>",
+ "<|special_1174|>",
+ "<|special_1175|>",
+ "<|special_1176|>",
+ "<|special_1177|>",
+ "<|special_1178|>",
+ "<|special_1179|>",
+ "<|special_1180|>",
+ "<|special_1181|>",
+ "<|special_1182|>",
+ "<|special_1183|>",
+ "<|special_1184|>",
+ "<|special_1185|>",
+ "<|special_1186|>",
+ "<|special_1187|>",
+ "<|special_1188|>",
+ "<|special_1189|>",
+ "<|special_1190|>",
+ "<|special_1191|>",
+ "<|special_1192|>",
+ "<|special_1193|>",
+ "<|special_1194|>",
+ "<|special_1195|>",
+ "<|special_1196|>",
+ "<|special_1197|>",
+ "<|special_1198|>",
+ "<|special_1199|>",
+ "<|special_1200|>",
+ "<|special_1201|>",
+ "<|special_1202|>",
+ "<|special_1203|>",
+ "<|special_1204|>",
+ "<|special_1205|>",
+ "<|special_1206|>",
+ "<|special_1207|>",
+ "<|special_1208|>",
+ "<|special_1209|>",
+ "<|special_1210|>",
+ "<|special_1211|>",
+ "<|special_1212|>",
+ "<|special_1213|>",
+ "<|special_1214|>",
+ "<|special_1215|>",
+ "<|special_1216|>",
+ "<|special_1217|>",
+ "<|special_1218|>",
+ "<|special_1219|>",
+ "<|special_1220|>",
+ "<|special_1221|>",
+ "<|special_1222|>",
+ "<|special_1223|>",
+ "<|special_1224|>",
+ "<|special_1225|>",
+ "<|special_1226|>",
+ "<|special_1227|>",
+ "<|special_1228|>",
+ "<|special_1229|>",
+ "<|special_1230|>",
+ "<|special_1231|>",
+ "<|special_1232|>",
+ "<|special_1233|>",
+ "<|special_1234|>",
+ "<|special_1235|>",
+ "<|special_1236|>",
+ "<|special_1237|>",
+ "<|special_1238|>",
+ "<|special_1239|>",
+ "<|special_1240|>",
+ "<|special_1241|>",
+ "<|special_1242|>",
+ "<|special_1243|>",
+ "<|special_1244|>",
+ "<|special_1245|>",
+ "<|special_1246|>",
+ "<|special_1247|>",
+ "<|special_1248|>",
+ "<|special_1249|>",
+ "<|special_1250|>",
+ "<|special_1251|>",
+ "<|special_1252|>",
+ "<|special_1253|>",
+ "<|special_1254|>",
+ "<|special_1255|>",
+ "<|special_1256|>",
+ "<|special_1257|>",
+ "<|special_1258|>",
+ "<|special_1259|>",
+ "<|special_1260|>",
+ "<|special_1261|>",
+ "<|special_1262|>",
+ "<|special_1263|>",
+ "<|special_1264|>",
+ "<|special_1265|>",
+ "<|special_1266|>",
+ "<|special_1267|>",
+ "<|special_1268|>",
+ "<|special_1269|>",
+ "<|special_1270|>",
+ "<|special_1271|>",
+ "<|special_1272|>",
+ "<|special_1273|>",
+ "<|special_1274|>",
+ "<|special_1275|>",
+ "<|special_1276|>",
+ "<|special_1277|>",
+ "<|special_1278|>",
+ "<|special_1279|>",
+ "<|special_1280|>",
+ "<|special_1281|>",
+ "<|special_1282|>",
+ "<|special_1283|>",
+ "<|special_1284|>",
+ "<|special_1285|>",
+ "<|special_1286|>",
+ "<|special_1287|>",
+ "<|special_1288|>",
+ "<|special_1289|>",
+ "<|special_1290|>",
+ "<|special_1291|>",
+ "<|special_1292|>",
+ "<|special_1293|>",
+ "<|special_1294|>",
+ "<|special_1295|>",
+ "<|special_1296|>",
+ "<|special_1297|>",
+ "<|special_1298|>",
+ "<|special_1299|>",
+ "<|special_1300|>",
+ "<|special_1301|>",
+ "<|special_1302|>",
+ "<|special_1303|>",
+ "<|special_1304|>",
+ "<|special_1305|>",
+ "<|special_1306|>",
+ "<|special_1307|>",
+ "<|special_1308|>",
+ "<|special_1309|>",
+ "<|special_1310|>",
+ "<|special_1311|>",
+ "<|special_1312|>",
+ "<|special_1313|>",
+ "<|special_1314|>",
+ "<|special_1315|>",
+ "<|special_1316|>",
+ "<|special_1317|>",
+ "<|special_1318|>",
+ "<|special_1319|>",
+ "<|special_1320|>",
+ "<|special_1321|>",
+ "<|special_1322|>",
+ "<|special_1323|>",
+ "<|special_1324|>",
+ "<|special_1325|>",
+ "<|special_1326|>",
+ "<|special_1327|>",
+ "<|special_1328|>",
+ "<|special_1329|>",
+ "<|special_1330|>",
+ "<|special_1331|>",
+ "<|special_1332|>",
+ "<|special_1333|>",
+ "<|special_1334|>",
+ "<|special_1335|>",
+ "<|special_1336|>",
+ "<|special_1337|>",
+ "<|special_1338|>",
+ "<|special_1339|>",
+ "<|special_1340|>",
+ "<|special_1341|>",
+ "<|special_1342|>",
+ "<|special_1343|>",
+ "<|special_1344|>",
+ "<|special_1345|>",
+ "<|special_1346|>",
+ "<|special_1347|>",
+ "<|special_1348|>",
+ "<|special_1349|>",
+ "<|special_1350|>",
+ "<|special_1351|>",
+ "<|special_1352|>",
+ "<|special_1353|>",
+ "<|special_1354|>",
+ "<|special_1355|>",
+ "<|special_1356|>",
+ "<|special_1357|>",
+ "<|special_1358|>",
+ "<|special_1359|>",
+ "<|special_1360|>",
+ "<|special_1361|>",
+ "<|special_1362|>",
+ "<|special_1363|>",
+ "<|special_1364|>",
+ "<|special_1365|>",
+ "<|special_1366|>",
+ "<|special_1367|>",
+ "<|special_1368|>",
+ "<|special_1369|>",
+ "<|special_1370|>",
+ "<|special_1371|>",
+ "<|special_1372|>",
+ "<|special_1373|>",
+ "<|special_1374|>",
+ "<|special_1375|>",
+ "<|special_1376|>",
+ "<|special_1377|>",
+ "<|special_1378|>",
+ "<|special_1379|>",
+ "<|special_1380|>",
+ "<|special_1381|>",
+ "<|special_1382|>",
+ "<|special_1383|>",
+ "<|special_1384|>",
+ "<|special_1385|>",
+ "<|special_1386|>",
+ "<|special_1387|>",
+ "<|special_1388|>",
+ "<|special_1389|>",
+ "<|special_1390|>",
+ "<|special_1391|>",
+ "<|special_1392|>",
+ "<|special_1393|>",
+ "<|special_1394|>",
+ "<|special_1395|>",
+ "<|special_1396|>",
+ "<|special_1397|>",
+ "<|special_1398|>",
+ "<|special_1399|>",
+ "<|special_1400|>",
+ "<|special_1401|>",
+ "<|special_1402|>",
+ "<|special_1403|>",
+ "<|special_1404|>",
+ "<|special_1405|>",
+ "<|special_1406|>",
+ "<|special_1407|>",
+ "<|special_1408|>",
+ "<|special_1409|>",
+ "<|special_1410|>",
+ "<|special_1411|>",
+ "<|special_1412|>",
+ "<|special_1413|>",
+ "<|special_1414|>",
+ "<|special_1415|>",
+ "<|special_1416|>",
+ "<|special_1417|>",
+ "<|special_1418|>",
+ "<|special_1419|>",
+ "<|special_1420|>",
+ "<|special_1421|>",
+ "<|special_1422|>",
+ "<|special_1423|>",
+ "<|special_1424|>",
+ "<|special_1425|>",
+ "<|special_1426|>",
+ "<|special_1427|>",
+ "<|special_1428|>",
+ "<|special_1429|>",
+ "<|special_1430|>",
+ "<|special_1431|>",
+ "<|special_1432|>",
+ "<|special_1433|>",
+ "<|special_1434|>",
+ "<|special_1435|>",
+ "<|special_1436|>",
+ "<|special_1437|>",
+ "<|special_1438|>",
+ "<|special_1439|>",
+ "<|special_1440|>",
+ "<|special_1441|>",
+ "<|special_1442|>",
+ "<|special_1443|>",
+ "<|special_1444|>",
+ "<|special_1445|>",
+ "<|special_1446|>",
+ "<|special_1447|>",
+ "<|special_1448|>",
+ "<|special_1449|>",
+ "<|special_1450|>",
+ "<|special_1451|>",
+ "<|special_1452|>",
+ "<|special_1453|>",
+ "<|special_1454|>",
+ "<|special_1455|>",
+ "<|special_1456|>",
+ "<|special_1457|>",
+ "<|special_1458|>",
+ "<|special_1459|>",
+ "<|special_1460|>",
+ "<|special_1461|>",
+ "<|special_1462|>",
+ "<|special_1463|>",
+ "<|special_1464|>",
+ "<|special_1465|>",
+ "<|special_1466|>",
+ "<|special_1467|>",
+ "<|special_1468|>",
+ "<|special_1469|>",
+ "<|special_1470|>",
+ "<|special_1471|>",
+ "<|special_1472|>",
+ "<|special_1473|>",
+ "<|special_1474|>",
+ "<|special_1475|>",
+ "<|special_1476|>",
+ "<|special_1477|>",
+ "<|special_1478|>",
+ "<|special_1479|>",
+ "<|special_1480|>",
+ "<|special_1481|>",
+ "<|special_1482|>",
+ "<|special_1483|>",
+ "<|special_1484|>",
+ "<|special_1485|>",
+ "<|special_1486|>",
+ "<|special_1487|>",
+ "<|special_1488|>",
+ "<|special_1489|>",
+ "<|special_1490|>",
+ "<|special_1491|>",
+ "<|special_1492|>",
+ "<|special_1493|>",
+ "<|special_1494|>",
+ "<|special_1495|>",
+ "<|special_1496|>",
+ "<|special_1497|>",
+ "<|special_1498|>",
+ "<|special_1499|>",
+ "<|special_1500|>",
+ "<|special_1501|>",
+ "<|special_1502|>",
+ "<|special_1503|>",
+ "<|special_1504|>",
+ "<|special_1505|>",
+ "<|special_1506|>",
+ "<|special_1507|>",
+ "<|special_1508|>",
+ "<|special_1509|>",
+ "<|special_1510|>",
+ "<|special_1511|>",
+ "<|special_1512|>",
+ "<|special_1513|>",
+ "<|special_1514|>",
+ "<|special_1515|>",
+ "<|special_1516|>",
+ "<|special_1517|>",
+ "<|special_1518|>",
+ "<|special_1519|>",
+ "<|special_1520|>",
+ "<|special_1521|>",
+ "<|special_1522|>",
+ "<|special_1523|>",
+ "<|special_1524|>",
+ "<|special_1525|>",
+ "<|special_1526|>",
+ "<|special_1527|>",
+ "<|special_1528|>",
+ "<|special_1529|>",
+ "<|special_1530|>",
+ "<|special_1531|>",
+ "<|special_1532|>",
+ "<|special_1533|>",
+ "<|special_1534|>",
+ "<|special_1535|>",
+ "<|special_1536|>",
+ "<|special_1537|>",
+ "<|special_1538|>",
+ "<|special_1539|>",
+ "<|special_1540|>",
+ "<|special_1541|>",
+ "<|special_1542|>",
+ "<|special_1543|>",
+ "<|special_1544|>",
+ "<|special_1545|>",
+ "<|special_1546|>",
+ "<|special_1547|>",
+ "<|special_1548|>",
+ "<|special_1549|>",
+ "<|special_1550|>",
+ "<|special_1551|>",
+ "<|special_1552|>",
+ "<|special_1553|>",
+ "<|special_1554|>",
+ "<|special_1555|>",
+ "<|special_1556|>",
+ "<|special_1557|>",
+ "<|special_1558|>",
+ "<|special_1559|>",
+ "<|special_1560|>",
+ "<|special_1561|>",
+ "<|special_1562|>",
+ "<|special_1563|>",
+ "<|special_1564|>",
+ "<|special_1565|>",
+ "<|special_1566|>",
+ "<|special_1567|>",
+ "<|special_1568|>",
+ "<|special_1569|>",
+ "<|special_1570|>",
+ "<|special_1571|>",
+ "<|special_1572|>",
+ "<|special_1573|>",
+ "<|special_1574|>",
+ "<|special_1575|>",
+ "<|special_1576|>",
+ "<|special_1577|>",
+ "<|special_1578|>",
+ "<|special_1579|>",
+ "<|special_1580|>",
+ "<|special_1581|>",
+ "<|special_1582|>",
+ "<|special_1583|>",
+ "<|special_1584|>",
+ "<|special_1585|>",
+ "<|special_1586|>",
+ "<|special_1587|>",
+ "<|special_1588|>",
+ "<|special_1589|>",
+ "<|special_1590|>",
+ "<|special_1591|>",
+ "<|special_1592|>",
+ "<|special_1593|>",
+ "<|special_1594|>",
+ "<|special_1595|>",
+ "<|special_1596|>",
+ "<|special_1597|>",
+ "<|special_1598|>",
+ "<|special_1599|>",
+ "<|special_1600|>",
+ "<|special_1601|>",
+ "<|special_1602|>",
+ "<|special_1603|>",
+ "<|special_1604|>",
+ "<|special_1605|>",
+ "<|special_1606|>",
+ "<|special_1607|>",
+ "<|special_1608|>",
+ "<|special_1609|>",
+ "<|special_1610|>",
+ "<|special_1611|>",
+ "<|special_1612|>",
+ "<|special_1613|>",
+ "<|special_1614|>",
+ "<|special_1615|>",
+ "<|special_1616|>",
+ "<|special_1617|>",
+ "<|special_1618|>",
+ "<|special_1619|>",
+ "<|special_1620|>",
+ "<|special_1621|>",
+ "<|special_1622|>",
+ "<|special_1623|>",
+ "<|special_1624|>",
+ "<|special_1625|>",
+ "<|special_1626|>",
+ "<|special_1627|>",
+ "<|special_1628|>",
+ "<|special_1629|>",
+ "<|special_1630|>",
+ "<|special_1631|>",
+ "<|special_1632|>",
+ "<|special_1633|>",
+ "<|special_1634|>",
+ "<|special_1635|>",
+ "<|special_1636|>",
+ "<|special_1637|>",
+ "<|special_1638|>",
+ "<|special_1639|>",
+ "<|special_1640|>",
+ "<|special_1641|>",
+ "<|special_1642|>",
+ "<|special_1643|>",
+ "<|special_1644|>",
+ "<|special_1645|>",
+ "<|special_1646|>",
+ "<|special_1647|>",
+ "<|special_1648|>",
+ "<|special_1649|>",
+ "<|special_1650|>",
+ "<|special_1651|>",
+ "<|special_1652|>",
+ "<|special_1653|>",
+ "<|special_1654|>",
+ "<|special_1655|>",
+ "<|special_1656|>",
+ "<|special_1657|>",
+ "<|special_1658|>",
+ "<|special_1659|>",
+ "<|special_1660|>",
+ "<|special_1661|>",
+ "<|special_1662|>",
+ "<|special_1663|>",
+ "<|special_1664|>",
+ "<|special_1665|>",
+ "<|special_1666|>",
+ "<|special_1667|>",
+ "<|special_1668|>",
+ "<|special_1669|>",
+ "<|special_1670|>",
+ "<|special_1671|>",
+ "<|special_1672|>",
+ "<|special_1673|>",
+ "<|special_1674|>",
+ "<|special_1675|>",
+ "<|special_1676|>",
+ "<|special_1677|>",
+ "<|special_1678|>",
+ "<|special_1679|>",
+ "<|special_1680|>",
+ "<|special_1681|>",
+ "<|special_1682|>",
+ "<|special_1683|>",
+ "<|special_1684|>",
+ "<|special_1685|>",
+ "<|special_1686|>",
+ "<|special_1687|>",
+ "<|special_1688|>",
+ "<|special_1689|>",
+ "<|special_1690|>",
+ "<|special_1691|>",
+ "<|special_1692|>",
+ "<|special_1693|>",
+ "<|special_1694|>",
+ "<|special_1695|>",
+ "<|special_1696|>",
+ "<|special_1697|>",
+ "<|special_1698|>",
+ "<|special_1699|>",
+ "<|special_1700|>",
+ "<|special_1701|>",
+ "<|special_1702|>",
+ "<|special_1703|>",
+ "<|special_1704|>",
+ "<|special_1705|>",
+ "<|special_1706|>",
+ "<|special_1707|>",
+ "<|special_1708|>",
+ "<|special_1709|>",
+ "<|special_1710|>",
+ "<|special_1711|>",
+ "<|special_1712|>",
+ "<|special_1713|>",
+ "<|special_1714|>",
+ "<|special_1715|>",
+ "<|special_1716|>",
+ "<|special_1717|>",
+ "<|special_1718|>",
+ "<|special_1719|>",
+ "<|special_1720|>",
+ "<|special_1721|>",
+ "<|special_1722|>",
+ "<|special_1723|>",
+ "<|special_1724|>",
+ "<|special_1725|>",
+ "<|special_1726|>",
+ "<|special_1727|>",
+ "<|special_1728|>",
+ "<|special_1729|>",
+ "<|special_1730|>",
+ "<|special_1731|>",
+ "<|special_1732|>",
+ "<|special_1733|>",
+ "<|special_1734|>",
+ "<|special_1735|>",
+ "<|special_1736|>",
+ "<|special_1737|>",
+ "<|special_1738|>",
+ "<|special_1739|>",
+ "<|special_1740|>",
+ "<|special_1741|>",
+ "<|special_1742|>",
+ "<|special_1743|>",
+ "<|special_1744|>",
+ "<|special_1745|>",
+ "<|special_1746|>",
+ "<|special_1747|>",
+ "<|special_1748|>",
+ "<|special_1749|>",
+ "<|special_1750|>",
+ "<|special_1751|>",
+ "<|special_1752|>",
+ "<|special_1753|>",
+ "<|special_1754|>",
+ "<|special_1755|>",
+ "<|special_1756|>",
+ "<|special_1757|>",
+ "<|special_1758|>",
+ "<|special_1759|>",
+ "<|special_1760|>",
+ "<|special_1761|>",
+ "<|special_1762|>",
+ "<|special_1763|>",
+ "<|special_1764|>",
+ "<|special_1765|>",
+ "<|special_1766|>",
+ "<|special_1767|>",
+ "<|special_1768|>",
+ "<|special_1769|>",
+ "<|special_1770|>",
+ "<|special_1771|>",
+ "<|special_1772|>",
+ "<|special_1773|>",
+ "<|special_1774|>",
+ "<|special_1775|>",
+ "<|special_1776|>",
+ "<|special_1777|>",
+ "<|special_1778|>",
+ "<|special_1779|>",
+ "<|special_1780|>",
+ "<|special_1781|>",
+ "<|special_1782|>",
+ "<|special_1783|>",
+ "<|special_1784|>",
+ "<|special_1785|>",
+ "<|special_1786|>",
+ "<|special_1787|>",
+ "<|special_1788|>",
+ "<|special_1789|>",
+ "<|special_1790|>",
+ "<|special_1791|>",
+ "<|special_1792|>",
+ "<|special_1793|>",
+ "<|special_1794|>",
+ "<|special_1795|>",
+ "<|special_1796|>",
+ "<|special_1797|>",
+ "<|special_1798|>",
+ "<|special_1799|>",
+ "<|special_1800|>",
+ "<|special_1801|>",
+ "<|special_1802|>",
+ "<|special_1803|>",
+ "<|special_1804|>",
+ "<|special_1805|>",
+ "<|special_1806|>",
+ "<|special_1807|>",
+ "<|special_1808|>",
+ "<|special_1809|>",
+ "<|special_1810|>",
+ "<|special_1811|>",
+ "<|special_1812|>",
+ "<|special_1813|>",
+ "<|special_1814|>",
+ "<|special_1815|>",
+ "<|special_1816|>",
+ "<|special_1817|>",
+ "<|special_1818|>",
+ "<|special_1819|>",
+ "<|special_1820|>",
+ "<|special_1821|>",
+ "<|special_1822|>",
+ "<|special_1823|>",
+ "<|special_1824|>",
+ "<|special_1825|>",
+ "<|special_1826|>",
+ "<|special_1827|>",
+ "<|special_1828|>",
+ "<|special_1829|>",
+ "<|special_1830|>",
+ "<|special_1831|>",
+ "<|special_1832|>",
+ "<|special_1833|>",
+ "<|special_1834|>",
+ "<|special_1835|>",
+ "<|special_1836|>",
+ "<|special_1837|>",
+ "<|special_1838|>",
+ "<|special_1839|>",
+ "<|special_1840|>",
+ "<|special_1841|>",
+ "<|special_1842|>",
+ "<|special_1843|>",
+ "<|special_1844|>",
+ "<|special_1845|>",
+ "<|special_1846|>",
+ "<|special_1847|>",
+ "<|special_1848|>",
+ "<|special_1849|>",
+ "<|special_1850|>",
+ "<|special_1851|>",
+ "<|special_1852|>",
+ "<|special_1853|>",
+ "<|special_1854|>",
+ "<|special_1855|>",
+ "<|special_1856|>",
+ "<|special_1857|>",
+ "<|special_1858|>",
+ "<|special_1859|>",
+ "<|special_1860|>",
+ "<|special_1861|>",
+ "<|special_1862|>",
+ "<|special_1863|>",
+ "<|special_1864|>",
+ "<|special_1865|>",
+ "<|special_1866|>",
+ "<|special_1867|>",
+ "<|special_1868|>",
+ "<|special_1869|>",
+ "<|special_1870|>",
+ "<|special_1871|>",
+ "<|special_1872|>",
+ "<|special_1873|>",
+ "<|special_1874|>",
+ "<|special_1875|>",
+ "<|special_1876|>",
+ "<|special_1877|>",
+ "<|special_1878|>",
+ "<|special_1879|>",
+ "<|special_1880|>",
+ "<|special_1881|>",
+ "<|special_1882|>",
+ "<|special_1883|>",
+ "<|special_1884|>",
+ "<|special_1885|>",
+ "<|special_1886|>",
+ "<|special_1887|>",
+ "<|special_1888|>",
+ "<|special_1889|>",
+ "<|special_1890|>",
+ "<|special_1891|>",
+ "<|special_1892|>",
+ "<|special_1893|>",
+ "<|special_1894|>",
+ "<|special_1895|>",
+ "<|special_1896|>",
+ "<|special_1897|>",
+ "<|special_1898|>",
+ "<|special_1899|>",
+ "<|special_1900|>",
+ "<|special_1901|>",
+ "<|special_1902|>",
+ "<|special_1903|>",
+ "<|special_1904|>",
+ "<|special_1905|>",
+ "<|special_1906|>",
+ "<|special_1907|>",
+ "<|special_1908|>",
+ "<|special_1909|>",
+ "<|special_1910|>",
+ "<|special_1911|>",
+ "<|special_1912|>",
+ "<|special_1913|>",
+ "<|special_1914|>",
+ "<|special_1915|>",
+ "<|special_1916|>",
+ "<|special_1917|>",
+ "<|special_1918|>",
+ "<|special_1919|>",
+ "<|special_1920|>",
+ "<|special_1921|>",
+ "<|special_1922|>",
+ "<|special_1923|>",
+ "<|special_1924|>",
+ "<|special_1925|>",
+ "<|special_1926|>",
+ "<|special_1927|>",
+ "<|special_1928|>",
+ "<|special_1929|>",
+ "<|special_1930|>",
+ "<|special_1931|>",
+ "<|special_1932|>",
+ "<|special_1933|>",
+ "<|special_1934|>",
+ "<|special_1935|>",
+ "<|special_1936|>",
+ "<|special_1937|>",
+ "<|special_1938|>",
+ "<|special_1939|>",
+ "<|special_1940|>",
+ "<|special_1941|>",
+ "<|special_1942|>",
+ "<|special_1943|>",
+ "<|special_1944|>",
+ "<|special_1945|>",
+ "<|special_1946|>",
+ "<|special_1947|>",
+ "<|special_1948|>",
+ "<|special_1949|>",
+ "<|special_1950|>",
+ "<|special_1951|>",
+ "<|special_1952|>",
+ "<|special_1953|>",
+ "<|special_1954|>",
+ "<|special_1955|>",
+ "<|special_1956|>",
+ "<|special_1957|>",
+ "<|special_1958|>",
+ "<|special_1959|>",
+ "<|special_1960|>",
+ "<|special_1961|>",
+ "<|special_1962|>",
+ "<|special_1963|>",
+ "<|special_1964|>",
+ "<|special_1965|>",
+ "<|special_1966|>",
+ "<|special_1967|>",
+ "<|special_1968|>",
+ "<|special_1969|>",
+ "<|special_1970|>",
+ "<|special_1971|>",
+ "<|special_1972|>",
+ "<|special_1973|>",
+ "<|special_1974|>",
+ "<|special_1975|>",
+ "<|special_1976|>",
+ "<|special_1977|>",
+ "<|special_1978|>",
+ "<|special_1979|>",
+ "<|special_1980|>",
+ "<|special_1981|>",
+ "<|special_1982|>",
+ "<|special_1983|>",
+ "<|special_1984|>",
+ "<|special_1985|>",
+ "<|special_1986|>",
+ "<|special_1987|>",
+ "<|special_1988|>",
+ "<|special_1989|>",
+ "<|special_1990|>",
+ "<|special_1991|>",
+ "<|special_1992|>",
+ "<|special_1993|>",
+ "<|special_1994|>",
+ "<|special_1995|>",
+ "<|special_1996|>",
+ "<|special_1997|>",
+ "<|special_1998|>",
+ "<|special_1999|>",
+ "<|special_2000|>",
+ "<|special_2001|>",
+ "<|special_2002|>",
+ "<|special_2003|>",
+ "<|special_2004|>",
+ "<|special_2005|>",
+ "<|special_2006|>",
+ "<|special_2007|>",
+ "<|special_2008|>",
+ "<|special_2009|>",
+ "<|special_2010|>",
+ "<|special_2011|>",
+ "<|special_2012|>",
+ "<|special_2013|>",
+ "<|special_2014|>",
+ "<|special_2015|>",
+ "<|special_2016|>",
+ "<|special_2017|>",
+ "<|special_2018|>",
+ "<|special_2019|>",
+ "<|special_2020|>",
+ "<|special_2021|>",
+ "<|special_2022|>",
+ "<|special_2023|>",
+ "<|special_2024|>",
+ "<|special_2025|>",
+ "<|special_2026|>",
+ "<|special_2027|>",
+ "<|special_2028|>",
+ "<|special_2029|>",
+ "<|special_2030|>",
+ "<|special_2031|>",
+ "<|special_2032|>",
+ "<|special_2033|>",
+ "<|special_2034|>",
+ "<|special_2035|>",
+ "<|special_2036|>",
+ "<|special_2037|>",
+ "<|special_2038|>",
+ "<|special_2039|>",
+ "<|special_2040|>",
+ "<|special_2041|>",
+ "<|special_2042|>",
+ "<|special_2043|>",
+ "<|special_2044|>",
+ "<|special_2045|>",
+ "<|special_2046|>",
+ "<|special_2047|>",
+ "<|special_2048|>",
+ "<|special_2049|>",
+ "<|special_2050|>",
+ "<|special_2051|>",
+ "<|special_2052|>",
+ "<|special_2053|>",
+ "<|special_2054|>",
+ "<|special_2055|>",
+ "<|special_2056|>",
+ "<|special_2057|>",
+ "<|special_2058|>",
+ "<|special_2059|>",
+ "<|special_2060|>",
+ "<|special_2061|>",
+ "<|special_2062|>",
+ "<|special_2063|>",
+ "<|special_2064|>",
+ "<|special_2065|>",
+ "<|special_2066|>",
+ "<|special_2067|>",
+ "<|special_2068|>",
+ "<|special_2069|>",
+ "<|special_2070|>",
+ "<|special_2071|>",
+ "<|special_2072|>",
+ "<|special_2073|>",
+ "<|special_2074|>",
+ "<|special_2075|>",
+ "<|special_2076|>",
+ "<|special_2077|>",
+ "<|special_2078|>",
+ "<|special_2079|>",
+ "<|special_2080|>",
+ "<|special_2081|>",
+ "<|special_2082|>",
+ "<|special_2083|>",
+ "<|special_2084|>",
+ "<|special_2085|>",
+ "<|special_2086|>",
+ "<|special_2087|>",
+ "<|special_2088|>",
+ "<|special_2089|>",
+ "<|special_2090|>",
+ "<|special_2091|>",
+ "<|special_2092|>",
+ "<|special_2093|>",
+ "<|special_2094|>",
+ "<|special_2095|>",
+ "<|special_2096|>",
+ "<|special_2097|>",
+ "<|special_2098|>",
+ "<|special_2099|>",
+ "<|special_2100|>",
+ "<|special_2101|>",
+ "<|special_2102|>",
+ "<|special_2103|>",
+ "<|special_2104|>",
+ "<|special_2105|>",
+ "<|special_2106|>",
+ "<|special_2107|>",
+ "<|special_2108|>",
+ "<|special_2109|>",
+ "<|special_2110|>",
+ "<|special_2111|>",
+ "<|special_2112|>",
+ "<|special_2113|>",
+ "<|special_2114|>",
+ "<|special_2115|>",
+ "<|special_2116|>",
+ "<|special_2117|>",
+ "<|special_2118|>",
+ "<|special_2119|>",
+ "<|special_2120|>",
+ "<|special_2121|>",
+ "<|special_2122|>",
+ "<|special_2123|>",
+ "<|special_2124|>",
+ "<|special_2125|>",
+ "<|special_2126|>",
+ "<|special_2127|>",
+ "<|special_2128|>",
+ "<|special_2129|>",
+ "<|special_2130|>",
+ "<|special_2131|>",
+ "<|special_2132|>",
+ "<|special_2133|>",
+ "<|special_2134|>",
+ "<|special_2135|>",
+ "<|special_2136|>",
+ "<|special_2137|>",
+ "<|special_2138|>",
+ "<|special_2139|>",
+ "<|special_2140|>",
+ "<|special_2141|>",
+ "<|special_2142|>",
+ "<|special_2143|>",
+ "<|special_2144|>",
+ "<|special_2145|>",
+ "<|special_2146|>",
+ "<|special_2147|>",
+ "<|special_2148|>",
+ "<|special_2149|>",
+ "<|special_2150|>",
+ "<|special_2151|>",
+ "<|special_2152|>",
+ "<|special_2153|>",
+ "<|special_2154|>",
+ "<|special_2155|>",
+ "<|special_2156|>",
+ "<|special_2157|>",
+ "<|special_2158|>",
+ "<|special_2159|>",
+ "<|special_2160|>",
+ "<|special_2161|>",
+ "<|special_2162|>",
+ "<|special_2163|>",
+ "<|special_2164|>",
+ "<|special_2165|>",
+ "<|special_2166|>",
+ "<|special_2167|>",
+ "<|special_2168|>",
+ "<|special_2169|>",
+ "<|special_2170|>",
+ "<|special_2171|>",
+ "<|special_2172|>",
+ "<|special_2173|>",
+ "<|special_2174|>",
+ "<|special_2175|>",
+ "<|special_2176|>",
+ "<|special_2177|>",
+ "<|special_2178|>",
+ "<|special_2179|>",
+ "<|special_2180|>",
+ "<|special_2181|>",
+ "<|special_2182|>",
+ "<|special_2183|>",
+ "<|special_2184|>",
+ "<|special_2185|>",
+ "<|special_2186|>",
+ "<|special_2187|>",
+ "<|special_2188|>",
+ "<|special_2189|>",
+ "<|special_2190|>",
+ "<|special_2191|>",
+ "<|special_2192|>",
+ "<|special_2193|>",
+ "<|special_2194|>",
+ "<|special_2195|>",
+ "<|special_2196|>",
+ "<|special_2197|>",
+ "<|special_2198|>",
+ "<|special_2199|>",
+ "<|special_2200|>",
+ "<|special_2201|>",
+ "<|special_2202|>",
+ "<|special_2203|>",
+ "<|special_2204|>",
+ "<|special_2205|>",
+ "<|special_2206|>",
+ "<|special_2207|>",
+ "<|special_2208|>",
+ "<|special_2209|>",
+ "<|special_2210|>",
+ "<|special_2211|>",
+ "<|special_2212|>",
+ "<|special_2213|>",
+ "<|special_2214|>",
+ "<|special_2215|>",
+ "<|special_2216|>",
+ "<|special_2217|>",
+ "<|special_2218|>",
+ "<|special_2219|>",
+ "<|special_2220|>",
+ "<|special_2221|>",
+ "<|special_2222|>",
+ "<|special_2223|>",
+ "<|special_2224|>",
+ "<|special_2225|>",
+ "<|special_2226|>",
+ "<|special_2227|>",
+ "<|special_2228|>",
+ "<|special_2229|>",
+ "<|special_2230|>",
+ "<|special_2231|>",
+ "<|special_2232|>",
+ "<|special_2233|>",
+ "<|special_2234|>",
+ "<|special_2235|>",
+ "<|special_2236|>",
+ "<|special_2237|>",
+ "<|special_2238|>",
+ "<|special_2239|>",
+ "<|special_2240|>",
+ "<|special_2241|>",
+ "<|special_2242|>",
+ "<|special_2243|>",
+ "<|special_2244|>",
+ "<|special_2245|>",
+ "<|special_2246|>",
+ "<|special_2247|>",
+ "<|special_2248|>",
+ "<|special_2249|>",
+ "<|special_2250|>",
+ "<|special_2251|>",
+ "<|special_2252|>",
+ "<|special_2253|>",
+ "<|special_2254|>",
+ "<|special_2255|>",
+ "<|special_2256|>",
+ "<|special_2257|>",
+ "<|special_2258|>",
+ "<|special_2259|>",
+ "<|special_2260|>",
+ "<|special_2261|>",
+ "<|special_2262|>",
+ "<|special_2263|>",
+ "<|special_2264|>",
+ "<|special_2265|>",
+ "<|special_2266|>",
+ "<|special_2267|>",
+ "<|special_2268|>",
+ "<|special_2269|>",
+ "<|special_2270|>",
+ "<|special_2271|>",
+ "<|special_2272|>",
+ "<|special_2273|>",
+ "<|special_2274|>",
+ "<|special_2275|>",
+ "<|special_2276|>",
+ "<|special_2277|>",
+ "<|special_2278|>",
+ "<|special_2279|>",
+ "<|special_2280|>",
+ "<|special_2281|>",
+ "<|special_2282|>",
+ "<|special_2283|>",
+ "<|special_2284|>",
+ "<|special_2285|>",
+ "<|special_2286|>",
+ "<|special_2287|>",
+ "<|special_2288|>",
+ "<|special_2289|>",
+ "<|special_2290|>",
+ "<|special_2291|>",
+ "<|special_2292|>",
+ "<|special_2293|>",
+ "<|special_2294|>",
+ "<|special_2295|>",
+ "<|special_2296|>",
+ "<|special_2297|>",
+ "<|special_2298|>",
+ "<|special_2299|>",
+ "<|special_2300|>",
+ "<|special_2301|>",
+ "<|special_2302|>",
+ "<|special_2303|>",
+ "<|special_2304|>",
+ "<|special_2305|>",
+ "<|special_2306|>",
+ "<|special_2307|>",
+ "<|special_2308|>",
+ "<|special_2309|>",
+ "<|special_2310|>",
+ "<|special_2311|>",
+ "<|special_2312|>",
+ "<|special_2313|>",
+ "<|special_2314|>",
+ "<|special_2315|>",
+ "<|special_2316|>",
+ "<|special_2317|>",
+ "<|special_2318|>",
+ "<|special_2319|>",
+ "<|special_2320|>",
+ "<|special_2321|>",
+ "<|special_2322|>",
+ "<|special_2323|>",
+ "<|special_2324|>",
+ "<|special_2325|>",
+ "<|special_2326|>",
+ "<|special_2327|>",
+ "<|special_2328|>",
+ "<|special_2329|>",
+ "<|special_2330|>",
+ "<|special_2331|>",
+ "<|special_2332|>",
+ "<|special_2333|>",
+ "<|special_2334|>",
+ "<|special_2335|>",
+ "<|special_2336|>",
+ "<|special_2337|>",
+ "<|special_2338|>",
+ "<|special_2339|>",
+ "<|special_2340|>",
+ "<|special_2341|>",
+ "<|special_2342|>",
+ "<|special_2343|>",
+ "<|special_2344|>",
+ "<|special_2345|>",
+ "<|special_2346|>",
+ "<|special_2347|>",
+ "<|special_2348|>",
+ "<|special_2349|>",
+ "<|special_2350|>",
+ "<|special_2351|>",
+ "<|special_2352|>",
+ "<|special_2353|>",
+ "<|special_2354|>",
+ "<|special_2355|>",
+ "<|special_2356|>",
+ "<|special_2357|>",
+ "<|special_2358|>",
+ "<|special_2359|>",
+ "<|special_2360|>",
+ "<|special_2361|>",
+ "<|special_2362|>",
+ "<|special_2363|>",
+ "<|special_2364|>",
+ "<|special_2365|>",
+ "<|special_2366|>",
+ "<|special_2367|>",
+ "<|special_2368|>",
+ "<|special_2369|>",
+ "<|special_2370|>",
+ "<|special_2371|>",
+ "<|special_2372|>",
+ "<|special_2373|>",
+ "<|special_2374|>",
+ "<|special_2375|>",
+ "<|special_2376|>",
+ "<|special_2377|>",
+ "<|special_2378|>",
+ "<|special_2379|>",
+ "<|special_2380|>",
+ "<|special_2381|>",
+ "<|special_2382|>",
+ "<|special_2383|>",
+ "<|special_2384|>",
+ "<|special_2385|>",
+ "<|special_2386|>",
+ "<|special_2387|>",
+ "<|special_2388|>",
+ "<|special_2389|>",
+ "<|special_2390|>",
+ "<|special_2391|>",
+ "<|special_2392|>",
+ "<|special_2393|>",
+ "<|special_2394|>",
+ "<|special_2395|>",
+ "<|special_2396|>",
+ "<|special_2397|>",
+ "<|special_2398|>",
+ "<|special_2399|>",
+ "<|special_2400|>",
+ "<|special_2401|>",
+ "<|special_2402|>",
+ "<|special_2403|>",
+ "<|special_2404|>",
+ "<|special_2405|>",
+ "<|special_2406|>",
+ "<|special_2407|>",
+ "<|special_2408|>",
+ "<|special_2409|>",
+ "<|special_2410|>",
+ "<|special_2411|>",
+ "<|special_2412|>",
+ "<|special_2413|>",
+ "<|special_2414|>",
+ "<|special_2415|>",
+ "<|special_2416|>",
+ "<|special_2417|>",
+ "<|special_2418|>",
+ "<|special_2419|>",
+ "<|special_2420|>",
+ "<|special_2421|>",
+ "<|special_2422|>",
+ "<|special_2423|>",
+ "<|special_2424|>",
+ "<|special_2425|>",
+ "<|special_2426|>",
+ "<|special_2427|>",
+ "<|special_2428|>",
+ "<|special_2429|>",
+ "<|special_2430|>",
+ "<|special_2431|>",
+ "<|special_2432|>",
+ "<|special_2433|>",
+ "<|special_2434|>",
+ "<|special_2435|>",
+ "<|special_2436|>",
+ "<|special_2437|>",
+ "<|special_2438|>",
+ "<|special_2439|>",
+ "<|special_2440|>",
+ "<|special_2441|>",
+ "<|special_2442|>",
+ "<|special_2443|>",
+ "<|special_2444|>",
+ "<|special_2445|>",
+ "<|special_2446|>",
+ "<|special_2447|>",
+ "<|special_2448|>",
+ "<|special_2449|>",
+ "<|special_2450|>",
+ "<|special_2451|>",
+ "<|special_2452|>",
+ "<|special_2453|>",
+ "<|special_2454|>",
+ "<|special_2455|>",
+ "<|special_2456|>",
+ "<|special_2457|>",
+ "<|special_2458|>",
+ "<|special_2459|>",
+ "<|special_2460|>",
+ "<|special_2461|>",
+ "<|special_2462|>",
+ "<|special_2463|>",
+ "<|special_2464|>",
+ "<|special_2465|>",
+ "<|special_2466|>",
+ "<|special_2467|>",
+ "<|special_2468|>",
+ "<|special_2469|>",
+ "<|special_2470|>",
+ "<|special_2471|>",
+ "<|special_2472|>",
+ "<|special_2473|>",
+ "<|special_2474|>",
+ "<|special_2475|>",
+ "<|special_2476|>",
+ "<|special_2477|>",
+ "<|special_2478|>",
+ "<|special_2479|>",
+ "<|special_2480|>",
+ "<|special_2481|>",
+ "<|special_2482|>",
+ "<|special_2483|>",
+ "<|special_2484|>",
+ "<|special_2485|>",
+ "<|special_2486|>",
+ "<|special_2487|>",
+ "<|special_2488|>",
+ "<|special_2489|>",
+ "<|special_2490|>",
+ "<|special_2491|>",
+ "<|special_2492|>",
+ "<|special_2493|>",
+ "<|special_2494|>",
+ "<|special_2495|>",
+ "<|special_2496|>",
+ "<|special_2497|>",
+ "<|special_2498|>",
+ "<|special_2499|>",
+ "<|special_2500|>",
+ "<|special_2501|>",
+ "<|special_2502|>",
+ "<|special_2503|>",
+ "<|special_2504|>",
+ "<|special_2505|>",
+ "<|special_2506|>",
+ "<|special_2507|>",
+ "<|special_2508|>",
+ "<|special_2509|>",
+ "<|special_2510|>",
+ "<|special_2511|>",
+ "<|special_2512|>",
+ "<|special_2513|>",
+ "<|special_2514|>",
+ "<|special_2515|>",
+ "<|special_2516|>",
+ "<|special_2517|>",
+ "<|special_2518|>",
+ "<|special_2519|>",
+ "<|special_2520|>",
+ "<|special_2521|>",
+ "<|special_2522|>",
+ "<|special_2523|>",
+ "<|special_2524|>",
+ "<|special_2525|>",
+ "<|special_2526|>",
+ "<|special_2527|>",
+ "<|special_2528|>",
+ "<|special_2529|>",
+ "<|special_2530|>",
+ "<|special_2531|>",
+ "<|special_2532|>",
+ "<|special_2533|>",
+ "<|special_2534|>",
+ "<|special_2535|>",
+ "<|special_2536|>",
+ "<|special_2537|>",
+ "<|special_2538|>",
+ "<|special_2539|>",
+ "<|special_2540|>",
+ "<|special_2541|>",
+ "<|special_2542|>",
+ "<|special_2543|>",
+ "<|special_2544|>",
+ "<|special_2545|>",
+ "<|special_2546|>",
+ "<|special_2547|>",
+ "<|special_2548|>",
+ "<|special_2549|>",
+ "<|special_2550|>",
+ "<|special_2551|>",
+ "<|special_2552|>",
+ "<|special_2553|>",
+ "<|special_2554|>",
+ "<|special_2555|>",
+ "<|special_2556|>",
+ "<|special_2557|>",
+ "<|special_2558|>",
+ "<|special_2559|>",
+ "<|special_2560|>",
+ "<|special_2561|>",
+ "<|special_2562|>",
+ "<|special_2563|>",
+ "<|special_2564|>",
+ "<|special_2565|>",
+ "<|special_2566|>",
+ "<|special_2567|>",
+ "<|special_2568|>",
+ "<|special_2569|>",
+ "<|special_2570|>",
+ "<|special_2571|>",
+ "<|special_2572|>",
+ "<|special_2573|>",
+ "<|special_2574|>",
+ "<|special_2575|>",
+ "<|special_2576|>",
+ "<|special_2577|>",
+ "<|special_2578|>",
+ "<|special_2579|>",
+ "<|special_2580|>",
+ "<|special_2581|>",
+ "<|special_2582|>",
+ "<|special_2583|>",
+ "<|special_2584|>",
+ "<|special_2585|>",
+ "<|special_2586|>",
+ "<|special_2587|>",
+ "<|special_2588|>",
+ "<|special_2589|>",
+ "<|special_2590|>",
+ "<|special_2591|>",
+ "<|special_2592|>",
+ "<|special_2593|>",
+ "<|special_2594|>",
+ "<|special_2595|>",
+ "<|special_2596|>",
+ "<|special_2597|>",
+ "<|special_2598|>",
+ "<|special_2599|>",
+ "<|special_2600|>",
+ "<|special_2601|>",
+ "<|special_2602|>",
+ "<|special_2603|>",
+ "<|special_2604|>",
+ "<|special_2605|>",
+ "<|special_2606|>",
+ "<|special_2607|>",
+ "<|special_2608|>",
+ "<|special_2609|>",
+ "<|special_2610|>",
+ "<|special_2611|>",
+ "<|special_2612|>",
+ "<|special_2613|>",
+ "<|special_2614|>",
+ "<|special_2615|>",
+ "<|special_2616|>",
+ "<|special_2617|>",
+ "<|special_2618|>",
+ "<|special_2619|>",
+ "<|special_2620|>",
+ "<|special_2621|>",
+ "<|special_2622|>",
+ "<|special_2623|>",
+ "<|special_2624|>",
+ "<|special_2625|>",
+ "<|special_2626|>",
+ "<|special_2627|>",
+ "<|special_2628|>",
+ "<|special_2629|>",
+ "<|special_2630|>",
+ "<|special_2631|>",
+ "<|special_2632|>",
+ "<|special_2633|>",
+ "<|special_2634|>",
+ "<|special_2635|>",
+ "<|special_2636|>",
+ "<|special_2637|>",
+ "<|special_2638|>",
+ "<|special_2639|>",
+ "<|special_2640|>",
+ "<|special_2641|>",
+ "<|special_2642|>",
+ "<|special_2643|>",
+ "<|special_2644|>",
+ "<|special_2645|>",
+ "<|special_2646|>",
+ "<|special_2647|>",
+ "<|special_2648|>",
+ "<|special_2649|>",
+ "<|special_2650|>",
+ "<|special_2651|>",
+ "<|special_2652|>",
+ "<|special_2653|>",
+ "<|special_2654|>",
+ "<|special_2655|>",
+ "<|special_2656|>",
+ "<|special_2657|>",
+ "<|special_2658|>",
+ "<|special_2659|>",
+ "<|special_2660|>",
+ "<|special_2661|>",
+ "<|special_2662|>",
+ "<|special_2663|>",
+ "<|special_2664|>",
+ "<|special_2665|>",
+ "<|special_2666|>",
+ "<|special_2667|>",
+ "<|special_2668|>",
+ "<|special_2669|>",
+ "<|special_2670|>",
+ "<|special_2671|>",
+ "<|special_2672|>",
+ "<|special_2673|>",
+ "<|special_2674|>",
+ "<|special_2675|>",
+ "<|special_2676|>",
+ "<|special_2677|>",
+ "<|special_2678|>",
+ "<|special_2679|>",
+ "<|special_2680|>",
+ "<|special_2681|>",
+ "<|special_2682|>",
+ "<|special_2683|>",
+ "<|special_2684|>",
+ "<|special_2685|>",
+ "<|special_2686|>",
+ "<|special_2687|>",
+ "<|special_2688|>",
+ "<|special_2689|>",
+ "<|special_2690|>",
+ "<|special_2691|>",
+ "<|special_2692|>",
+ "<|special_2693|>",
+ "<|special_2694|>",
+ "<|special_2695|>",
+ "<|special_2696|>",
+ "<|special_2697|>",
+ "<|special_2698|>",
+ "<|special_2699|>",
+ "<|special_2700|>",
+ "<|special_2701|>",
+ "<|special_2702|>",
+ "<|special_2703|>",
+ "<|special_2704|>",
+ "<|special_2705|>",
+ "<|special_2706|>",
+ "<|special_2707|>",
+ "<|special_2708|>",
+ "<|special_2709|>",
+ "<|special_2710|>",
+ "<|special_2711|>",
+ "<|special_2712|>",
+ "<|special_2713|>",
+ "<|special_2714|>",
+ "<|special_2715|>",
+ "<|special_2716|>",
+ "<|special_2717|>",
+ "<|special_2718|>",
+ "<|special_2719|>",
+ "<|special_2720|>",
+ "<|special_2721|>",
+ "<|special_2722|>",
+ "<|special_2723|>",
+ "<|special_2724|>",
+ "<|special_2725|>",
+ "<|special_2726|>",
+ "<|special_2727|>",
+ "<|special_2728|>",
+ "<|special_2729|>",
+ "<|special_2730|>",
+ "<|special_2731|>",
+ "<|special_2732|>",
+ "<|special_2733|>",
+ "<|special_2734|>",
+ "<|special_2735|>",
+ "<|special_2736|>",
+ "<|special_2737|>",
+ "<|special_2738|>",
+ "<|special_2739|>",
+ "<|special_2740|>",
+ "<|special_2741|>",
+ "<|special_2742|>",
+ "<|special_2743|>",
+ "<|special_2744|>",
+ "<|special_2745|>",
+ "<|special_2746|>",
+ "<|special_2747|>",
+ "<|special_2748|>",
+ "<|special_2749|>",
+ "<|special_2750|>",
+ "<|special_2751|>",
+ "<|special_2752|>",
+ "<|special_2753|>",
+ "<|special_2754|>",
+ "<|special_2755|>",
+ "<|special_2756|>",
+ "<|special_2757|>",
+ "<|special_2758|>",
+ "<|special_2759|>",
+ "<|special_2760|>",
+ "<|special_2761|>",
+ "<|special_2762|>",
+ "<|special_2763|>",
+ "<|special_2764|>",
+ "<|special_2765|>",
+ "<|special_2766|>",
+ "<|special_2767|>",
+ "<|special_2768|>",
+ "<|special_2769|>",
+ "<|special_2770|>",
+ "<|special_2771|>",
+ "<|special_2772|>",
+ "<|special_2773|>",
+ "<|special_2774|>",
+ "<|special_2775|>",
+ "<|special_2776|>",
+ "<|special_2777|>",
+ "<|special_2778|>",
+ "<|special_2779|>",
+ "<|special_2780|>",
+ "<|special_2781|>",
+ "<|special_2782|>",
+ "<|special_2783|>",
+ "<|special_2784|>",
+ "<|special_2785|>",
+ "<|special_2786|>",
+ "<|special_2787|>",
+ "<|special_2788|>",
+ "<|special_2789|>",
+ "<|special_2790|>",
+ "<|special_2791|>",
+ "<|special_2792|>",
+ "<|special_2793|>",
+ "<|special_2794|>",
+ "<|special_2795|>",
+ "<|special_2796|>",
+ "<|special_2797|>",
+ "<|special_2798|>",
+ "<|special_2799|>",
+ "<|special_2800|>",
+ "<|special_2801|>",
+ "<|special_2802|>",
+ "<|special_2803|>",
+ "<|special_2804|>",
+ "<|special_2805|>",
+ "<|special_2806|>",
+ "<|special_2807|>",
+ "<|special_2808|>",
+ "<|special_2809|>",
+ "<|special_2810|>",
+ "<|special_2811|>",
+ "<|special_2812|>",
+ "<|special_2813|>",
+ "<|special_2814|>",
+ "<|special_2815|>",
+ "<|special_2816|>",
+ "<|special_2817|>",
+ "<|special_2818|>",
+ "<|special_2819|>",
+ "<|special_2820|>",
+ "<|special_2821|>",
+ "<|special_2822|>",
+ "<|special_2823|>",
+ "<|special_2824|>",
+ "<|special_2825|>",
+ "<|special_2826|>",
+ "<|special_2827|>",
+ "<|special_2828|>",
+ "<|special_2829|>",
+ "<|special_2830|>",
+ "<|special_2831|>",
+ "<|special_2832|>",
+ "<|special_2833|>",
+ "<|special_2834|>",
+ "<|special_2835|>",
+ "<|special_2836|>",
+ "<|special_2837|>",
+ "<|special_2838|>",
+ "<|special_2839|>",
+ "<|special_2840|>",
+ "<|special_2841|>",
+ "<|special_2842|>",
+ "<|special_2843|>",
+ "<|special_2844|>",
+ "<|special_2845|>",
+ "<|special_2846|>",
+ "<|special_2847|>",
+ "<|special_2848|>",
+ "<|special_2849|>",
+ "<|special_2850|>",
+ "<|special_2851|>",
+ "<|special_2852|>",
+ "<|special_2853|>",
+ "<|special_2854|>",
+ "<|special_2855|>",
+ "<|special_2856|>",
+ "<|special_2857|>",
+ "<|special_2858|>",
+ "<|special_2859|>",
+ "<|special_2860|>",
+ "<|special_2861|>",
+ "<|special_2862|>",
+ "<|special_2863|>",
+ "<|special_2864|>",
+ "<|special_2865|>",
+ "<|special_2866|>",
+ "<|special_2867|>",
+ "<|special_2868|>",
+ "<|special_2869|>",
+ "<|special_2870|>",
+ "<|special_2871|>",
+ "<|special_2872|>",
+ "<|special_2873|>",
+ "<|special_2874|>",
+ "<|special_2875|>",
+ "<|special_2876|>",
+ "<|special_2877|>",
+ "<|special_2878|>",
+ "<|special_2879|>",
+ "<|special_2880|>",
+ "<|special_2881|>",
+ "<|special_2882|>",
+ "<|special_2883|>",
+ "<|special_2884|>",
+ "<|special_2885|>",
+ "<|special_2886|>",
+ "<|special_2887|>",
+ "<|special_2888|>",
+ "<|special_2889|>",
+ "<|special_2890|>",
+ "<|special_2891|>",
+ "<|special_2892|>",
+ "<|special_2893|>",
+ "<|special_2894|>",
+ "<|special_2895|>",
+ "<|special_2896|>",
+ "<|special_2897|>",
+ "<|special_2898|>",
+ "<|special_2899|>",
+ "<|special_2900|>",
+ "<|special_2901|>",
+ "<|special_2902|>",
+ "<|special_2903|>",
+ "<|special_2904|>",
+ "<|special_2905|>",
+ "<|special_2906|>",
+ "<|special_2907|>",
+ "<|special_2908|>",
+ "<|special_2909|>",
+ "<|special_2910|>",
+ "<|special_2911|>",
+ "<|special_2912|>",
+ "<|special_2913|>",
+ "<|special_2914|>",
+ "<|special_2915|>",
+ "<|special_2916|>",
+ "<|special_2917|>",
+ "<|special_2918|>",
+ "<|special_2919|>",
+ "<|special_2920|>",
+ "<|special_2921|>",
+ "<|special_2922|>",
+ "<|special_2923|>",
+ "<|special_2924|>",
+ "<|special_2925|>",
+ "<|special_2926|>",
+ "<|special_2927|>",
+ "<|special_2928|>",
+ "<|special_2929|>",
+ "<|special_2930|>",
+ "<|special_2931|>",
+ "<|special_2932|>",
+ "<|special_2933|>",
+ "<|special_2934|>",
+ "<|special_2935|>",
+ "<|special_2936|>",
+ "<|special_2937|>",
+ "<|special_2938|>",
+ "<|special_2939|>",
+ "<|special_2940|>",
+ "<|special_2941|>",
+ "<|special_2942|>",
+ "<|special_2943|>",
+ "<|special_2944|>",
+ "<|special_2945|>",
+ "<|special_2946|>",
+ "<|special_2947|>",
+ "<|special_2948|>",
+ "<|special_2949|>",
+ "<|special_2950|>",
+ "<|special_2951|>",
+ "<|special_2952|>",
+ "<|special_2953|>",
+ "<|special_2954|>",
+ "<|special_2955|>",
+ "<|special_2956|>",
+ "<|special_2957|>",
+ "<|special_2958|>",
+ "<|special_2959|>",
+ "<|special_2960|>",
+ "<|special_2961|>",
+ "<|special_2962|>",
+ "<|special_2963|>",
+ "<|special_2964|>",
+ "<|special_2965|>",
+ "<|special_2966|>",
+ "<|special_2967|>",
+ "<|special_2968|>",
+ "<|special_2969|>",
+ "<|special_2970|>",
+ "<|special_2971|>",
+ "<|special_2972|>",
+ "<|special_2973|>",
+ "<|special_2974|>",
+ "<|special_2975|>",
+ "<|special_2976|>",
+ "<|special_2977|>",
+ "<|special_2978|>",
+ "<|special_2979|>",
+ "<|special_2980|>",
+ "<|special_2981|>",
+ "<|special_2982|>",
+ "<|special_2983|>",
+ "<|special_2984|>",
+ "<|special_2985|>",
+ "<|special_2986|>",
+ "<|special_2987|>",
+ "<|special_2988|>",
+ "<|special_2989|>",
+ "<|special_2990|>",
+ "<|special_2991|>",
+ "<|special_2992|>",
+ "<|special_2993|>",
+ "<|special_2994|>",
+ "<|special_2995|>",
+ "<|special_2996|>",
+ "<|special_2997|>",
+ "<|special_2998|>",
+ "<|special_2999|>",
+ "<|special_3000|>",
+ "<|special_3001|>",
+ "<|special_3002|>",
+ "<|special_3003|>",
+ "<|special_3004|>",
+ "<|special_3005|>",
+ "<|special_3006|>",
+ "<|special_3007|>",
+ "<|special_3008|>",
+ "<|special_3009|>",
+ "<|special_3010|>",
+ "<|special_3011|>",
+ "<|special_3012|>",
+ "<|special_3013|>",
+ "<|special_3014|>",
+ "<|special_3015|>",
+ "<|special_3016|>",
+ "<|special_3017|>",
+ "<|special_3018|>",
+ "<|special_3019|>",
+ "<|special_3020|>",
+ "<|special_3021|>",
+ "<|special_3022|>",
+ "<|special_3023|>",
+ "<|special_3024|>",
+ "<|special_3025|>",
+ "<|special_3026|>",
+ "<|special_3027|>",
+ "<|special_3028|>",
+ "<|special_3029|>",
+ "<|special_3030|>",
+ "<|special_3031|>",
+ "<|special_3032|>",
+ "<|special_3033|>",
+ "<|special_3034|>",
+ "<|special_3035|>",
+ "<|special_3036|>",
+ "<|special_3037|>",
+ "<|special_3038|>",
+ "<|special_3039|>",
+ "<|special_3040|>",
+ "<|special_3041|>",
+ "<|special_3042|>",
+ "<|special_3043|>",
+ "<|special_3044|>",
+ "<|special_3045|>",
+ "<|special_3046|>",
+ "<|special_3047|>",
+ "<|special_3048|>",
+ "<|special_3049|>",
+ "<|special_3050|>",
+ "<|special_3051|>",
+ "<|special_3052|>",
+ "<|special_3053|>",
+ "<|special_3054|>",
+ "<|special_3055|>",
+ "<|special_3056|>",
+ "<|special_3057|>",
+ "<|special_3058|>",
+ "<|special_3059|>",
+ "<|special_3060|>",
+ "<|special_3061|>",
+ "<|special_3062|>",
+ "<|special_3063|>",
+ "<|special_3064|>",
+ "<|special_3065|>",
+ "<|special_3066|>",
+ "<|special_3067|>",
+ "<|special_3068|>",
+ "<|special_3069|>",
+ "<|special_3070|>",
+ "<|special_3071|>",
+ "<|special_3072|>",
+ "<|special_3073|>",
+ "<|special_3074|>",
+ "<|special_3075|>",
+ "<|special_3076|>",
+ "<|special_3077|>",
+ "<|special_3078|>",
+ "<|special_3079|>",
+ "<|special_3080|>",
+ "<|special_3081|>",
+ "<|special_3082|>",
+ "<|special_3083|>",
+ "<|special_3084|>",
+ "<|special_3085|>",
+ "<|special_3086|>",
+ "<|special_3087|>",
+ "<|special_3088|>",
+ "<|special_3089|>",
+ "<|special_3090|>",
+ "<|special_3091|>",
+ "<|special_3092|>",
+ "<|special_3093|>",
+ "<|special_3094|>",
+ "<|special_3095|>",
+ "<|special_3096|>",
+ "<|special_3097|>",
+ "<|special_3098|>",
+ "<|special_3099|>",
+ "<|special_3100|>",
+ "<|special_3101|>",
+ "<|special_3102|>",
+ "<|special_3103|>",
+ "<|special_3104|>",
+ "<|special_3105|>",
+ "<|special_3106|>",
+ "<|special_3107|>",
+ "<|special_3108|>",
+ "<|special_3109|>",
+ "<|special_3110|>",
+ "<|special_3111|>",
+ "<|special_3112|>",
+ "<|special_3113|>",
+ "<|special_3114|>",
+ "<|special_3115|>",
+ "<|special_3116|>",
+ "<|special_3117|>",
+ "<|special_3118|>",
+ "<|special_3119|>",
+ "<|special_3120|>",
+ "<|special_3121|>",
+ "<|special_3122|>",
+ "<|special_3123|>",
+ "<|special_3124|>",
+ "<|special_3125|>",
+ "<|special_3126|>",
+ "<|special_3127|>",
+ "<|special_3128|>",
+ "<|special_3129|>",
+ "<|special_3130|>",
+ "<|special_3131|>",
+ "<|special_3132|>",
+ "<|special_3133|>",
+ "<|special_3134|>",
+ "<|special_3135|>",
+ "<|special_3136|>",
+ "<|special_3137|>",
+ "<|special_3138|>",
+ "<|special_3139|>",
+ "<|special_3140|>",
+ "<|special_3141|>",
+ "<|special_3142|>",
+ "<|special_3143|>",
+ "<|special_3144|>",
+ "<|special_3145|>",
+ "<|special_3146|>",
+ "<|special_3147|>",
+ "<|special_3148|>",
+ "<|special_3149|>",
+ "<|special_3150|>",
+ "<|special_3151|>",
+ "<|special_3152|>",
+ "<|special_3153|>",
+ "<|special_3154|>",
+ "<|special_3155|>",
+ "<|special_3156|>",
+ "<|special_3157|>",
+ "<|special_3158|>",
+ "<|special_3159|>",
+ "<|special_3160|>",
+ "<|special_3161|>",
+ "<|special_3162|>",
+ "<|special_3163|>",
+ "<|special_3164|>",
+ "<|special_3165|>",
+ "<|special_3166|>",
+ "<|special_3167|>",
+ "<|special_3168|>",
+ "<|special_3169|>",
+ "<|special_3170|>",
+ "<|special_3171|>",
+ "<|special_3172|>",
+ "<|special_3173|>",
+ "<|special_3174|>",
+ "<|special_3175|>",
+ "<|special_3176|>",
+ "<|special_3177|>",
+ "<|special_3178|>",
+ "<|special_3179|>",
+ "<|special_3180|>",
+ "<|special_3181|>",
+ "<|special_3182|>",
+ "<|special_3183|>",
+ "<|special_3184|>",
+ "<|special_3185|>",
+ "<|special_3186|>",
+ "<|special_3187|>",
+ "<|special_3188|>",
+ "<|special_3189|>",
+ "<|special_3190|>",
+ "<|special_3191|>",
+ "<|special_3192|>",
+ "<|special_3193|>",
+ "<|special_3194|>",
+ "<|special_3195|>",
+ "<|special_3196|>",
+ "<|special_3197|>",
+ "<|special_3198|>",
+ "<|special_3199|>",
+ "<|special_3200|>",
+ "<|special_3201|>",
+ "<|special_3202|>",
+ "<|special_3203|>",
+ "<|special_3204|>",
+ "<|special_3205|>",
+ "<|special_3206|>",
+ "<|special_3207|>",
+ "<|special_3208|>",
+ "<|special_3209|>",
+ "<|special_3210|>",
+ "<|special_3211|>",
+ "<|special_3212|>",
+ "<|special_3213|>",
+ "<|special_3214|>",
+ "<|special_3215|>",
+ "<|special_3216|>",
+ "<|special_3217|>",
+ "<|special_3218|>",
+ "<|special_3219|>",
+ "<|special_3220|>",
+ "<|special_3221|>",
+ "<|special_3222|>",
+ "<|special_3223|>",
+ "<|special_3224|>",
+ "<|special_3225|>",
+ "<|special_3226|>",
+ "<|special_3227|>",
+ "<|special_3228|>",
+ "<|special_3229|>",
+ "<|special_3230|>",
+ "<|special_3231|>",
+ "<|special_3232|>",
+ "<|special_3233|>",
+ "<|special_3234|>",
+ "<|special_3235|>",
+ "<|special_3236|>",
+ "<|special_3237|>",
+ "<|special_3238|>",
+ "<|special_3239|>",
+ "<|special_3240|>",
+ "<|special_3241|>",
+ "<|special_3242|>",
+ "<|special_3243|>",
+ "<|special_3244|>",
+ "<|special_3245|>",
+ "<|special_3246|>",
+ "<|special_3247|>",
+ "<|special_3248|>",
+ "<|special_3249|>",
+ "<|special_3250|>",
+ "<|special_3251|>",
+ "<|special_3252|>",
+ "<|special_3253|>",
+ "<|special_3254|>",
+ "<|special_3255|>",
+ "<|special_3256|>",
+ "<|special_3257|>",
+ "<|special_3258|>",
+ "<|special_3259|>",
+ "<|special_3260|>",
+ "<|special_3261|>",
+ "<|special_3262|>",
+ "<|special_3263|>",
+ "<|special_3264|>",
+ "<|special_3265|>",
+ "<|special_3266|>",
+ "<|special_3267|>",
+ "<|special_3268|>",
+ "<|special_3269|>",
+ "<|special_3270|>",
+ "<|special_3271|>",
+ "<|special_3272|>",
+ "<|special_3273|>",
+ "<|special_3274|>",
+ "<|special_3275|>",
+ "<|special_3276|>",
+ "<|special_3277|>",
+ "<|special_3278|>",
+ "<|special_3279|>",
+ "<|special_3280|>",
+ "<|special_3281|>",
+ "<|special_3282|>",
+ "<|special_3283|>",
+ "<|special_3284|>",
+ "<|special_3285|>",
+ "<|special_3286|>",
+ "<|special_3287|>",
+ "<|special_3288|>",
+ "<|special_3289|>",
+ "<|special_3290|>",
+ "<|special_3291|>",
+ "<|special_3292|>",
+ "<|special_3293|>",
+ "<|special_3294|>",
+ "<|special_3295|>",
+ "<|special_3296|>",
+ "<|special_3297|>",
+ "<|special_3298|>",
+ "<|special_3299|>",
+ "<|special_3300|>",
+ "<|special_3301|>",
+ "<|special_3302|>",
+ "<|special_3303|>",
+ "<|special_3304|>",
+ "<|special_3305|>",
+ "<|special_3306|>",
+ "<|special_3307|>",
+ "<|special_3308|>",
+ "<|special_3309|>",
+ "<|special_3310|>",
+ "<|special_3311|>",
+ "<|special_3312|>",
+ "<|special_3313|>",
+ "<|special_3314|>",
+ "<|special_3315|>",
+ "<|special_3316|>",
+ "<|special_3317|>",
+ "<|special_3318|>",
+ "<|special_3319|>",
+ "<|special_3320|>",
+ "<|special_3321|>",
+ "<|special_3322|>",
+ "<|special_3323|>",
+ "<|special_3324|>",
+ "<|special_3325|>",
+ "<|special_3326|>",
+ "<|special_3327|>",
+ "<|special_3328|>",
+ "<|special_3329|>",
+ "<|special_3330|>",
+ "<|special_3331|>",
+ "<|special_3332|>",
+ "<|special_3333|>",
+ "<|special_3334|>",
+ "<|special_3335|>",
+ "<|special_3336|>",
+ "<|special_3337|>",
+ "<|special_3338|>",
+ "<|special_3339|>",
+ "<|special_3340|>",
+ "<|special_3341|>",
+ "<|special_3342|>",
+ "<|special_3343|>",
+ "<|special_3344|>",
+ "<|special_3345|>",
+ "<|special_3346|>",
+ "<|special_3347|>",
+ "<|special_3348|>",
+ "<|special_3349|>",
+ "<|special_3350|>",
+ "<|special_3351|>",
+ "<|special_3352|>",
+ "<|special_3353|>",
+ "<|special_3354|>",
+ "<|special_3355|>",
+ "<|special_3356|>",
+ "<|special_3357|>",
+ "<|special_3358|>",
+ "<|special_3359|>",
+ "<|special_3360|>",
+ "<|special_3361|>",
+ "<|special_3362|>",
+ "<|special_3363|>",
+ "<|special_3364|>",
+ "<|special_3365|>",
+ "<|special_3366|>",
+ "<|special_3367|>",
+ "<|special_3368|>",
+ "<|special_3369|>",
+ "<|special_3370|>",
+ "<|special_3371|>",
+ "<|special_3372|>",
+ "<|special_3373|>",
+ "<|special_3374|>",
+ "<|special_3375|>",
+ "<|special_3376|>",
+ "<|special_3377|>",
+ "<|special_3378|>",
+ "<|special_3379|>",
+ "<|special_3380|>",
+ "<|special_3381|>",
+ "<|special_3382|>",
+ "<|special_3383|>",
+ "<|special_3384|>",
+ "<|special_3385|>",
+ "<|special_3386|>",
+ "<|special_3387|>",
+ "<|special_3388|>",
+ "<|special_3389|>",
+ "<|special_3390|>",
+ "<|special_3391|>",
+ "<|special_3392|>",
+ "<|special_3393|>",
+ "<|special_3394|>",
+ "<|special_3395|>",
+ "<|special_3396|>",
+ "<|special_3397|>",
+ "<|special_3398|>",
+ "<|special_3399|>",
+ "<|special_3400|>",
+ "<|special_3401|>",
+ "<|special_3402|>",
+ "<|special_3403|>",
+ "<|special_3404|>",
+ "<|special_3405|>",
+ "<|special_3406|>",
+ "<|special_3407|>",
+ "<|special_3408|>",
+ "<|special_3409|>",
+ "<|special_3410|>",
+ "<|special_3411|>",
+ "<|special_3412|>",
+ "<|special_3413|>",
+ "<|special_3414|>",
+ "<|special_3415|>",
+ "<|special_3416|>",
+ "<|special_3417|>",
+ "<|special_3418|>",
+ "<|special_3419|>",
+ "<|special_3420|>",
+ "<|special_3421|>",
+ "<|special_3422|>",
+ "<|special_3423|>",
+ "<|special_3424|>",
+ "<|special_3425|>",
+ "<|special_3426|>",
+ "<|special_3427|>",
+ "<|special_3428|>",
+ "<|special_3429|>",
+ "<|special_3430|>",
+ "<|special_3431|>",
+ "<|special_3432|>",
+ "<|special_3433|>",
+ "<|special_3434|>",
+ "<|special_3435|>",
+ "<|special_3436|>",
+ "<|special_3437|>",
+ "<|special_3438|>",
+ "<|special_3439|>",
+ "<|special_3440|>",
+ "<|special_3441|>",
+ "<|special_3442|>",
+ "<|special_3443|>",
+ "<|special_3444|>",
+ "<|special_3445|>",
+ "<|special_3446|>",
+ "<|special_3447|>",
+ "<|special_3448|>",
+ "<|special_3449|>",
+ "<|special_3450|>",
+ "<|special_3451|>",
+ "<|special_3452|>",
+ "<|special_3453|>",
+ "<|special_3454|>",
+ "<|special_3455|>",
+ "<|special_3456|>",
+ "<|special_3457|>",
+ "<|special_3458|>",
+ "<|special_3459|>",
+ "<|special_3460|>",
+ "<|special_3461|>",
+ "<|special_3462|>",
+ "<|special_3463|>",
+ "<|special_3464|>",
+ "<|special_3465|>",
+ "<|special_3466|>",
+ "<|special_3467|>",
+ "<|special_3468|>",
+ "<|special_3469|>",
+ "<|special_3470|>",
+ "<|special_3471|>",
+ "<|special_3472|>",
+ "<|special_3473|>",
+ "<|special_3474|>",
+ "<|special_3475|>",
+ "<|special_3476|>",
+ "<|special_3477|>",
+ "<|special_3478|>",
+ "<|special_3479|>",
+ "<|special_3480|>",
+ "<|special_3481|>",
+ "<|special_3482|>",
+ "<|special_3483|>",
+ "<|special_3484|>",
+ "<|special_3485|>",
+ "<|special_3486|>",
+ "<|special_3487|>",
+ "<|special_3488|>",
+ "<|special_3489|>",
+ "<|special_3490|>",
+ "<|special_3491|>",
+ "<|special_3492|>",
+ "<|special_3493|>",
+ "<|special_3494|>",
+ "<|special_3495|>",
+ "<|special_3496|>",
+ "<|special_3497|>",
+ "<|special_3498|>",
+ "<|special_3499|>",
+ "<|special_3500|>",
+ "<|special_3501|>",
+ "<|special_3502|>",
+ "<|special_3503|>",
+ "<|special_3504|>",
+ "<|special_3505|>",
+ "<|special_3506|>",
+ "<|special_3507|>",
+ "<|special_3508|>",
+ "<|special_3509|>",
+ "<|special_3510|>",
+ "<|special_3511|>",
+ "<|special_3512|>",
+ "<|special_3513|>",
+ "<|special_3514|>",
+ "<|special_3515|>",
+ "<|special_3516|>",
+ "<|special_3517|>",
+ "<|special_3518|>",
+ "<|special_3519|>",
+ "<|special_3520|>",
+ "<|special_3521|>",
+ "<|special_3522|>",
+ "<|special_3523|>",
+ "<|special_3524|>",
+ "<|special_3525|>",
+ "<|special_3526|>",
+ "<|special_3527|>",
+ "<|special_3528|>",
+ "<|special_3529|>",
+ "<|special_3530|>",
+ "<|special_3531|>",
+ "<|special_3532|>",
+ "<|special_3533|>",
+ "<|special_3534|>",
+ "<|special_3535|>",
+ "<|special_3536|>",
+ "<|special_3537|>",
+ "<|special_3538|>",
+ "<|special_3539|>",
+ "<|special_3540|>",
+ "<|special_3541|>",
+ "<|special_3542|>",
+ "<|special_3543|>",
+ "<|special_3544|>",
+ "<|special_3545|>",
+ "<|special_3546|>",
+ "<|special_3547|>",
+ "<|special_3548|>",
+ "<|special_3549|>",
+ "<|special_3550|>",
+ "<|special_3551|>",
+ "<|special_3552|>",
+ "<|special_3553|>",
+ "<|special_3554|>",
+ "<|special_3555|>",
+ "<|special_3556|>",
+ "<|special_3557|>",
+ "<|special_3558|>",
+ "<|special_3559|>",
+ "<|special_3560|>",
+ "<|special_3561|>",
+ "<|special_3562|>",
+ "<|special_3563|>",
+ "<|special_3564|>",
+ "<|special_3565|>",
+ "<|special_3566|>",
+ "<|special_3567|>",
+ "<|special_3568|>",
+ "<|special_3569|>",
+ "<|special_3570|>",
+ "<|special_3571|>",
+ "<|special_3572|>",
+ "<|special_3573|>",
+ "<|special_3574|>",
+ "<|special_3575|>",
+ "<|special_3576|>",
+ "<|special_3577|>",
+ "<|special_3578|>",
+ "<|special_3579|>",
+ "<|special_3580|>",
+ "<|special_3581|>",
+ "<|special_3582|>",
+ "<|special_3583|>",
+ "<|special_3584|>",
+ "<|special_3585|>",
+ "<|special_3586|>",
+ "<|special_3587|>",
+ "<|special_3588|>",
+ "<|special_3589|>",
+ "<|special_3590|>",
+ "<|special_3591|>",
+ "<|special_3592|>",
+ "<|special_3593|>",
+ "<|special_3594|>",
+ "<|special_3595|>",
+ "<|special_3596|>",
+ "<|special_3597|>",
+ "<|special_3598|>",
+ "<|special_3599|>",
+ "<|special_3600|>",
+ "<|special_3601|>",
+ "<|special_3602|>",
+ "<|special_3603|>",
+ "<|special_3604|>",
+ "<|special_3605|>",
+ "<|special_3606|>",
+ "<|special_3607|>",
+ "<|special_3608|>",
+ "<|special_3609|>",
+ "<|special_3610|>",
+ "<|special_3611|>",
+ "<|special_3612|>",
+ "<|special_3613|>",
+ "<|special_3614|>",
+ "<|special_3615|>",
+ "<|special_3616|>",
+ "<|special_3617|>",
+ "<|special_3618|>",
+ "<|special_3619|>",
+ "<|special_3620|>",
+ "<|special_3621|>",
+ "<|special_3622|>",
+ "<|special_3623|>",
+ "<|special_3624|>",
+ "<|special_3625|>",
+ "<|special_3626|>",
+ "<|special_3627|>",
+ "<|special_3628|>",
+ "<|special_3629|>",
+ "<|special_3630|>",
+ "<|special_3631|>",
+ "<|special_3632|>",
+ "<|special_3633|>",
+ "<|special_3634|>",
+ "<|special_3635|>",
+ "<|special_3636|>",
+ "<|special_3637|>",
+ "<|special_3638|>",
+ "<|special_3639|>",
+ "<|special_3640|>",
+ "<|special_3641|>",
+ "<|special_3642|>",
+ "<|special_3643|>",
+ "<|special_3644|>",
+ "<|special_3645|>",
+ "<|special_3646|>",
+ "<|special_3647|>",
+ "<|special_3648|>",
+ "<|special_3649|>",
+ "<|special_3650|>",
+ "<|special_3651|>",
+ "<|special_3652|>",
+ "<|special_3653|>",
+ "<|special_3654|>",
+ "<|special_3655|>",
+ "<|special_3656|>",
+ "<|special_3657|>",
+ "<|special_3658|>",
+ "<|special_3659|>",
+ "<|special_3660|>",
+ "<|special_3661|>",
+ "<|special_3662|>",
+ "<|special_3663|>",
+ "<|special_3664|>",
+ "<|special_3665|>",
+ "<|special_3666|>",
+ "<|special_3667|>",
+ "<|special_3668|>",
+ "<|special_3669|>",
+ "<|special_3670|>",
+ "<|special_3671|>",
+ "<|special_3672|>",
+ "<|special_3673|>",
+ "<|special_3674|>",
+ "<|special_3675|>",
+ "<|special_3676|>",
+ "<|special_3677|>",
+ "<|special_3678|>",
+ "<|special_3679|>",
+ "<|special_3680|>",
+ "<|special_3681|>",
+ "<|special_3682|>",
+ "<|special_3683|>",
+ "<|special_3684|>",
+ "<|special_3685|>",
+ "<|special_3686|>",
+ "<|special_3687|>",
+ "<|special_3688|>",
+ "<|special_3689|>",
+ "<|special_3690|>",
+ "<|special_3691|>",
+ "<|special_3692|>",
+ "<|special_3693|>",
+ "<|special_3694|>",
+ "<|special_3695|>",
+ "<|special_3696|>",
+ "<|special_3697|>",
+ "<|special_3698|>",
+ "<|special_3699|>",
+ "<|special_3700|>",
+ "<|special_3701|>",
+ "<|special_3702|>",
+ "<|special_3703|>",
+ "<|special_3704|>",
+ "<|special_3705|>",
+ "<|special_3706|>",
+ "<|special_3707|>",
+ "<|special_3708|>",
+ "<|special_3709|>",
+ "<|special_3710|>",
+ "<|special_3711|>",
+ "<|special_3712|>",
+ "<|special_3713|>",
+ "<|special_3714|>",
+ "<|special_3715|>",
+ "<|special_3716|>",
+ "<|special_3717|>",
+ "<|special_3718|>",
+ "<|special_3719|>",
+ "<|special_3720|>",
+ "<|special_3721|>",
+ "<|special_3722|>",
+ "<|special_3723|>",
+ "<|special_3724|>",
+ "<|special_3725|>",
+ "<|special_3726|>",
+ "<|special_3727|>",
+ "<|special_3728|>",
+ "<|special_3729|>",
+ "<|special_3730|>",
+ "<|special_3731|>",
+ "<|special_3732|>",
+ "<|special_3733|>",
+ "<|special_3734|>",
+ "<|special_3735|>",
+ "<|special_3736|>",
+ "<|special_3737|>",
+ "<|special_3738|>",
+ "<|special_3739|>",
+ "<|special_3740|>",
+ "<|special_3741|>",
+ "<|special_3742|>",
+ "<|special_3743|>",
+ "<|special_3744|>",
+ "<|special_3745|>",
+ "<|special_3746|>",
+ "<|special_3747|>",
+ "<|special_3748|>",
+ "<|special_3749|>",
+ "<|special_3750|>",
+ "<|special_3751|>",
+ "<|special_3752|>",
+ "<|special_3753|>",
+ "<|special_3754|>",
+ "<|special_3755|>",
+ "<|special_3756|>",
+ "<|special_3757|>",
+ "<|special_3758|>",
+ "<|special_3759|>",
+ "<|special_3760|>",
+ "<|special_3761|>",
+ "<|special_3762|>",
+ "<|special_3763|>",
+ "<|special_3764|>",
+ "<|special_3765|>",
+ "<|special_3766|>",
+ "<|special_3767|>",
+ "<|special_3768|>",
+ "<|special_3769|>",
+ "<|special_3770|>",
+ "<|special_3771|>",
+ "<|special_3772|>",
+ "<|special_3773|>",
+ "<|special_3774|>",
+ "<|special_3775|>",
+ "<|special_3776|>",
+ "<|special_3777|>",
+ "<|special_3778|>",
+ "<|special_3779|>",
+ "<|special_3780|>",
+ "<|special_3781|>",
+ "<|special_3782|>",
+ "<|special_3783|>",
+ "<|special_3784|>",
+ "<|special_3785|>",
+ "<|special_3786|>",
+ "<|special_3787|>",
+ "<|special_3788|>",
+ "<|special_3789|>",
+ "<|special_3790|>",
+ "<|special_3791|>",
+ "<|special_3792|>",
+ "<|special_3793|>",
+ "<|special_3794|>",
+ "<|special_3795|>",
+ "<|special_3796|>",
+ "<|special_3797|>",
+ "<|special_3798|>",
+ "<|special_3799|>",
+ "<|special_3800|>",
+ "<|special_3801|>",
+ "<|special_3802|>",
+ "<|special_3803|>",
+ "<|special_3804|>",
+ "<|special_3805|>",
+ "<|special_3806|>",
+ "<|special_3807|>",
+ "<|special_3808|>",
+ "<|special_3809|>",
+ "<|special_3810|>",
+ "<|special_3811|>",
+ "<|special_3812|>",
+ "<|special_3813|>",
+ "<|special_3814|>",
+ "<|special_3815|>",
+ "<|special_3816|>",
+ "<|special_3817|>",
+ "<|special_3818|>",
+ "<|special_3819|>",
+ "<|special_3820|>",
+ "<|special_3821|>",
+ "<|special_3822|>",
+ "<|special_3823|>",
+ "<|special_3824|>",
+ "<|special_3825|>",
+ "<|special_3826|>",
+ "<|special_3827|>",
+ "<|special_3828|>",
+ "<|special_3829|>",
+ "<|special_3830|>",
+ "<|special_3831|>",
+ "<|special_3832|>",
+ "<|special_3833|>",
+ "<|special_3834|>",
+ "<|special_3835|>",
+ "<|special_3836|>",
+ "<|special_3837|>",
+ "<|special_3838|>",
+ "<|special_3839|>",
+ "<|special_3840|>",
+ "<|special_3841|>",
+ "<|special_3842|>",
+ "<|special_3843|>",
+ "<|special_3844|>",
+ "<|special_3845|>",
+ "<|special_3846|>",
+ "<|special_3847|>",
+ "<|special_3848|>",
+ "<|special_3849|>",
+ "<|special_3850|>",
+ "<|special_3851|>",
+ "<|special_3852|>",
+ "<|special_3853|>",
+ "<|special_3854|>",
+ "<|special_3855|>",
+ "<|special_3856|>",
+ "<|special_3857|>",
+ "<|special_3858|>",
+ "<|special_3859|>",
+ "<|special_3860|>",
+ "<|special_3861|>",
+ "<|special_3862|>",
+ "<|special_3863|>",
+ "<|special_3864|>",
+ "<|special_3865|>",
+ "<|special_3866|>",
+ "<|special_3867|>",
+ "<|special_3868|>",
+ "<|special_3869|>",
+ "<|special_3870|>",
+ "<|special_3871|>",
+ "<|special_3872|>",
+ "<|special_3873|>",
+ "<|special_3874|>",
+ "<|special_3875|>",
+ "<|special_3876|>",
+ "<|special_3877|>",
+ "<|special_3878|>",
+ "<|special_3879|>",
+ "<|special_3880|>",
+ "<|special_3881|>",
+ "<|special_3882|>",
+ "<|special_3883|>",
+ "<|special_3884|>",
+ "<|special_3885|>",
+ "<|special_3886|>",
+ "<|special_3887|>",
+ "<|special_3888|>",
+ "<|special_3889|>",
+ "<|special_3890|>",
+ "<|special_3891|>",
+ "<|special_3892|>",
+ "<|special_3893|>",
+ "<|special_3894|>",
+ "<|special_3895|>",
+ "<|special_3896|>",
+ "<|special_3897|>",
+ "<|special_3898|>",
+ "<|special_3899|>",
+ "<|special_3900|>",
+ "<|special_3901|>",
+ "<|special_3902|>",
+ "<|special_3903|>",
+ "<|special_3904|>",
+ "<|special_3905|>",
+ "<|special_3906|>",
+ "<|special_3907|>",
+ "<|special_3908|>",
+ "<|special_3909|>",
+ "<|special_3910|>",
+ "<|special_3911|>",
+ "<|special_3912|>",
+ "<|special_3913|>",
+ "<|special_3914|>",
+ "<|special_3915|>",
+ "<|special_3916|>",
+ "<|special_3917|>",
+ "<|special_3918|>",
+ "<|special_3919|>",
+ "<|special_3920|>",
+ "<|special_3921|>",
+ "<|special_3922|>",
+ "<|special_3923|>",
+ "<|special_3924|>",
+ "<|special_3925|>",
+ "<|special_3926|>",
+ "<|special_3927|>",
+ "<|special_3928|>",
+ "<|special_3929|>",
+ "<|special_3930|>",
+ "<|special_3931|>",
+ "<|special_3932|>",
+ "<|special_3933|>",
+ "<|special_3934|>",
+ "<|special_3935|>",
+ "<|special_3936|>",
+ "<|special_3937|>",
+ "<|special_3938|>",
+ "<|special_3939|>",
+ "<|special_3940|>",
+ "<|special_3941|>",
+ "<|special_3942|>",
+ "<|special_3943|>",
+ "<|special_3944|>",
+ "<|special_3945|>",
+ "<|special_3946|>",
+ "<|special_3947|>",
+ "<|special_3948|>",
+ "<|special_3949|>",
+ "<|special_3950|>",
+ "<|special_3951|>",
+ "<|special_3952|>",
+ "<|special_3953|>",
+ "<|special_3954|>",
+ "<|special_3955|>",
+ "<|special_3956|>",
+ "<|special_3957|>",
+ "<|special_3958|>",
+ "<|special_3959|>",
+ "<|special_3960|>",
+ "<|special_3961|>",
+ "<|special_3962|>",
+ "<|special_3963|>",
+ "<|special_3964|>",
+ "<|special_3965|>",
+ "<|special_3966|>",
+ "<|special_3967|>",
+ "<|special_3968|>",
+ "<|special_3969|>",
+ "<|special_3970|>",
+ "<|special_3971|>",
+ "<|special_3972|>",
+ "<|special_3973|>",
+ "<|special_3974|>",
+ "<|special_3975|>",
+ "<|special_3976|>",
+ "<|special_3977|>",
+ "<|special_3978|>",
+ "<|special_3979|>",
+ "<|special_3980|>",
+ "<|special_3981|>",
+ "<|special_3982|>",
+ "<|special_3983|>",
+ "<|special_3984|>",
+ "<|special_3985|>",
+ "<|special_3986|>",
+ "<|special_3987|>",
+ "<|special_3988|>",
+ "<|special_3989|>",
+ "<|special_3990|>",
+ "<|special_3991|>",
+ "<|special_3992|>",
+ "<|special_3993|>",
+ "<|special_3994|>",
+ "<|special_3995|>",
+ "<|special_3996|>",
+ "<|special_3997|>",
+ "<|special_3998|>",
+ "<|special_3999|>",
+ "<|special_4000|>",
+ "<|special_4001|>",
+ "<|special_4002|>",
+ "<|special_4003|>",
+ "<|special_4004|>",
+ "<|special_4005|>",
+ "<|special_4006|>",
+ "<|special_4007|>",
+ "<|special_4008|>",
+ "<|special_4009|>",
+ "<|special_4010|>",
+ "<|special_4011|>",
+ "<|special_4012|>",
+ "<|special_4013|>",
+ "<|special_4014|>",
+ "<|special_4015|>",
+ "<|special_4016|>",
+ "<|special_4017|>",
+ "<|special_4018|>",
+ "<|special_4019|>",
+ "<|special_4020|>",
+ "<|special_4021|>",
+ "<|special_4022|>",
+ "<|special_4023|>",
+ "<|special_4024|>",
+ "<|special_4025|>",
+ "<|special_4026|>",
+ "<|special_4027|>",
+ "<|special_4028|>",
+ "<|special_4029|>",
+ "<|special_4030|>",
+ "<|special_4031|>",
+ "<|special_4032|>",
+ "<|special_4033|>",
+ "<|special_4034|>",
+ "<|special_4035|>",
+ "<|special_4036|>",
+ "<|special_4037|>",
+ "<|special_4038|>",
+ "<|special_4039|>",
+ "<|special_4040|>",
+ "<|special_4041|>",
+ "<|special_4042|>",
+ "<|special_4043|>",
+ "<|special_4044|>",
+ "<|special_4045|>",
+ "<|special_4046|>",
+ "<|special_4047|>",
+ "<|special_4048|>",
+ "<|special_4049|>",
+ "<|special_4050|>",
+ "<|special_4051|>",
+ "<|special_4052|>",
+ "<|special_4053|>",
+ "<|special_4054|>",
+ "<|special_4055|>",
+ "<|special_4056|>",
+ "<|special_4057|>",
+ "<|special_4058|>",
+ "<|special_4059|>",
+ "<|special_4060|>",
+ "<|special_4061|>",
+ "<|special_4062|>",
+ "<|special_4063|>",
+ "<|special_4064|>",
+ "<|special_4065|>",
+ "<|special_4066|>",
+ "<|special_4067|>",
+ "<|special_4068|>",
+ "<|special_4069|>",
+ "<|special_4070|>",
+ "<|special_4071|>",
+ "<|special_4072|>",
+ "<|special_4073|>",
+ "<|special_4074|>",
+ "<|special_4075|>",
+ "<|special_4076|>",
+ "<|special_4077|>",
+ "<|special_4078|>",
+ "<|special_4079|>",
+ "<|special_4080|>",
+ "<|special_4081|>",
+ "<|special_4082|>",
+ "<|special_4083|>",
+ "<|special_4084|>",
+ "<|special_4085|>",
+ "<|special_4086|>",
+ "<|special_4087|>",
+ "<|special_4088|>",
+ "<|special_4089|>",
+ "<|special_4090|>",
+ "<|special_4091|>",
+ "<|special_4092|>",
+ "<|special_4093|>",
+ "<|special_4094|>",
+ "<|special_4095|>"
+ ],
+ "bos_token": {
+ "content": "<|startoftext|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "eos_token": {
+ "content": "<|endoftext|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "pad_token": {
+ "content": "<|endoftext|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ },
+ "unk_token": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false
+ }
+}
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..4efa523f65ed330de970ca54e6d7e416651267a0
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7a8c81670e4651cc290fbe0ab397e0357abfa933964b9b6f56f508557c04c19
+size 16473485
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..ce9c8bfee5baf89222f7ef9f6038d10852db1841
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,36762 @@
+{
+ "add_bos_token": false,
+ "add_eos_token": false,
+ "added_tokens_decoder": {
+ "0": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "1": {
+ "content": "<|startoftext|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "2": {
+ "content": "<|endoftext|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "3": {
+ "content": "<|fim_prefix|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "4": {
+ "content": "<|fim_middle|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "5": {
+ "content": "<|fim_suffix|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "6": {
+ "content": "<|special_6|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "7": {
+ "content": "<|special_7|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "8": {
+ "content": "<|special_8|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "9": {
+ "content": "<|special_9|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "10": {
+ "content": "<|special_10|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "11": {
+ "content": "<|special_11|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "12": {
+ "content": "<|special_12|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "13": {
+ "content": "<|special_13|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "14": {
+ "content": "<|special_14|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "15": {
+ "content": "<|special_15|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "16": {
+ "content": "<|special_16|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "17": {
+ "content": "<|special_17|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "18": {
+ "content": "<|special_18|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "19": {
+ "content": "<|special_19|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "20": {
+ "content": "<|begin|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "21": {
+ "content": "<|end|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "22": {
+ "content": "<|think|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "23": {
+ "content": "<|content|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "24": {
+ "content": "<|flush|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "25": {
+ "content": "<|calls|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "26": {
+ "content": "<|tools:begin|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "27": {
+ "content": "<|tools:end|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "28": {
+ "content": "<|tool:begin|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "29": {
+ "content": "<|tool:end|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "30": {
+ "content": "<|tool_calls|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "31": {
+ "content": "<|tool_call:begin|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "32": {
+ "content": "<|tool_call:end|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "33": {
+ "content": "<|tool_call:name|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "34": {
+ "content": "<|tool_call:args|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "35": {
+ "content": "<|tool_response|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "36": {
+ "content": "<|tool_response:begin|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "37": {
+ "content": "<|tool_response:end|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "38": {
+ "content": "<|tool_response:name|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "39": {
+ "content": "<|tool_response:result|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "40": {
+ "content": "<|special_40|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "41": {
+ "content": "<|special_41|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "42": {
+ "content": "<|special_42|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "43": {
+ "content": "<|special_43|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "44": {
+ "content": "<|special_44|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "45": {
+ "content": "<|special_45|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "46": {
+ "content": "<|special_46|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "47": {
+ "content": "<|special_47|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "48": {
+ "content": "<|special_48|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "49": {
+ "content": "<|special_49|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "50": {
+ "content": "<|special_50|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "51": {
+ "content": "<|special_51|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "52": {
+ "content": "<|special_52|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "53": {
+ "content": "<|special_53|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "54": {
+ "content": "<|special_54|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "55": {
+ "content": "<|special_55|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "56": {
+ "content": "<|special_56|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "57": {
+ "content": "<|special_57|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "58": {
+ "content": "<|special_58|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "59": {
+ "content": "<|special_59|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "60": {
+ "content": "<|special_60|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "61": {
+ "content": "<|special_61|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "62": {
+ "content": "<|special_62|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "63": {
+ "content": "<|special_63|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "64": {
+ "content": "<|special_64|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "65": {
+ "content": "<|special_65|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "66": {
+ "content": "<|special_66|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "67": {
+ "content": "<|special_67|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "68": {
+ "content": "<|special_68|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "69": {
+ "content": "<|special_69|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "70": {
+ "content": "<|special_70|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "71": {
+ "content": "<|special_71|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "72": {
+ "content": "<|special_72|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "73": {
+ "content": "<|special_73|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "74": {
+ "content": "<|special_74|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "75": {
+ "content": "<|special_75|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "76": {
+ "content": "<|special_76|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "77": {
+ "content": "<|special_77|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "78": {
+ "content": "<|special_78|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "79": {
+ "content": "<|special_79|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "80": {
+ "content": "<|special_80|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "81": {
+ "content": "<|special_81|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "82": {
+ "content": "<|special_82|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "83": {
+ "content": "<|special_83|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "84": {
+ "content": "<|special_84|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "85": {
+ "content": "<|special_85|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "86": {
+ "content": "<|special_86|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "87": {
+ "content": "<|special_87|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "88": {
+ "content": "<|special_88|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "89": {
+ "content": "<|special_89|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "90": {
+ "content": "<|special_90|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "91": {
+ "content": "<|special_91|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "92": {
+ "content": "<|special_92|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "93": {
+ "content": "<|special_93|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "94": {
+ "content": "<|special_94|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "95": {
+ "content": "<|special_95|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "96": {
+ "content": "<|special_96|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "97": {
+ "content": "<|special_97|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "98": {
+ "content": "<|special_98|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "99": {
+ "content": "<|special_99|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "100": {
+ "content": "<|special_100|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "101": {
+ "content": "<|special_101|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "102": {
+ "content": "<|special_102|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "103": {
+ "content": "<|special_103|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "104": {
+ "content": "<|special_104|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "105": {
+ "content": "<|special_105|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "106": {
+ "content": "<|special_106|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "107": {
+ "content": "<|special_107|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "108": {
+ "content": "<|special_108|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "109": {
+ "content": "<|special_109|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "110": {
+ "content": "<|special_110|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "111": {
+ "content": "<|special_111|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "112": {
+ "content": "<|special_112|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "113": {
+ "content": "<|special_113|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "114": {
+ "content": "<|special_114|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "115": {
+ "content": "<|special_115|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "116": {
+ "content": "<|special_116|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "117": {
+ "content": "<|special_117|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "118": {
+ "content": "<|special_118|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "119": {
+ "content": "<|special_119|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "120": {
+ "content": "<|special_120|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "121": {
+ "content": "<|special_121|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "122": {
+ "content": "<|special_122|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "123": {
+ "content": "<|special_123|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "124": {
+ "content": "<|special_124|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "125": {
+ "content": "<|special_125|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "126": {
+ "content": "<|special_126|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "127": {
+ "content": "<|special_127|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "128": {
+ "content": "<|special_128|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "129": {
+ "content": "<|special_129|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "130": {
+ "content": "<|special_130|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "131": {
+ "content": "<|special_131|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "132": {
+ "content": "<|special_132|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "133": {
+ "content": "<|special_133|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "134": {
+ "content": "<|special_134|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "135": {
+ "content": "<|special_135|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "136": {
+ "content": "<|special_136|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "137": {
+ "content": "<|special_137|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "138": {
+ "content": "<|special_138|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "139": {
+ "content": "<|special_139|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "140": {
+ "content": "<|special_140|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "141": {
+ "content": "<|special_141|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "142": {
+ "content": "<|special_142|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "143": {
+ "content": "<|special_143|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "144": {
+ "content": "<|special_144|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "145": {
+ "content": "<|special_145|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "146": {
+ "content": "<|special_146|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "147": {
+ "content": "<|special_147|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "148": {
+ "content": "<|special_148|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "149": {
+ "content": "<|special_149|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "150": {
+ "content": "<|special_150|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "151": {
+ "content": "<|special_151|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "152": {
+ "content": "<|special_152|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "153": {
+ "content": "<|special_153|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "154": {
+ "content": "<|special_154|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "155": {
+ "content": "<|special_155|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "156": {
+ "content": "<|special_156|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "157": {
+ "content": "<|special_157|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "158": {
+ "content": "<|special_158|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "159": {
+ "content": "<|special_159|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "160": {
+ "content": "<|special_160|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "161": {
+ "content": "<|special_161|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "162": {
+ "content": "<|special_162|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "163": {
+ "content": "<|special_163|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "164": {
+ "content": "<|special_164|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "165": {
+ "content": "<|special_165|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "166": {
+ "content": "<|special_166|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "167": {
+ "content": "<|special_167|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "168": {
+ "content": "<|special_168|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "169": {
+ "content": "<|special_169|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "170": {
+ "content": "<|special_170|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "171": {
+ "content": "<|special_171|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "172": {
+ "content": "<|special_172|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "173": {
+ "content": "<|special_173|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "174": {
+ "content": "<|special_174|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "175": {
+ "content": "<|special_175|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "176": {
+ "content": "<|special_176|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "177": {
+ "content": "<|special_177|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "178": {
+ "content": "<|special_178|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "179": {
+ "content": "<|special_179|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "180": {
+ "content": "<|special_180|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "181": {
+ "content": "<|special_181|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "182": {
+ "content": "<|special_182|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "183": {
+ "content": "<|special_183|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "184": {
+ "content": "<|special_184|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "185": {
+ "content": "<|special_185|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "186": {
+ "content": "<|special_186|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "187": {
+ "content": "<|special_187|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "188": {
+ "content": "<|special_188|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "189": {
+ "content": "<|special_189|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "190": {
+ "content": "<|special_190|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "191": {
+ "content": "<|special_191|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "192": {
+ "content": "<|special_192|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "193": {
+ "content": "<|special_193|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "194": {
+ "content": "<|special_194|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "195": {
+ "content": "<|special_195|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "196": {
+ "content": "<|special_196|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "197": {
+ "content": "<|special_197|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "198": {
+ "content": "<|special_198|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "199": {
+ "content": "<|special_199|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "200": {
+ "content": "<|special_200|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "201": {
+ "content": "<|special_201|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "202": {
+ "content": "<|special_202|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "203": {
+ "content": "<|special_203|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "204": {
+ "content": "<|special_204|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "205": {
+ "content": "<|special_205|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "206": {
+ "content": "<|special_206|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "207": {
+ "content": "<|special_207|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "208": {
+ "content": "<|special_208|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "209": {
+ "content": "<|special_209|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "210": {
+ "content": "<|special_210|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "211": {
+ "content": "<|special_211|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "212": {
+ "content": "<|special_212|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "213": {
+ "content": "<|special_213|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "214": {
+ "content": "<|special_214|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "215": {
+ "content": "<|special_215|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "216": {
+ "content": "<|special_216|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "217": {
+ "content": "<|special_217|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "218": {
+ "content": "<|special_218|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "219": {
+ "content": "<|special_219|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "220": {
+ "content": "<|special_220|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "221": {
+ "content": "<|special_221|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "222": {
+ "content": "<|special_222|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "223": {
+ "content": "<|special_223|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "224": {
+ "content": "<|special_224|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "225": {
+ "content": "<|special_225|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "226": {
+ "content": "<|special_226|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "227": {
+ "content": "<|special_227|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "228": {
+ "content": "<|special_228|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "229": {
+ "content": "<|special_229|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "230": {
+ "content": "<|special_230|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "231": {
+ "content": "<|special_231|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "232": {
+ "content": "<|special_232|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "233": {
+ "content": "<|special_233|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "234": {
+ "content": "<|special_234|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "235": {
+ "content": "<|special_235|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "236": {
+ "content": "<|special_236|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "237": {
+ "content": "<|special_237|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "238": {
+ "content": "<|special_238|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "239": {
+ "content": "<|special_239|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "240": {
+ "content": "<|special_240|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "241": {
+ "content": "<|special_241|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "242": {
+ "content": "<|special_242|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "243": {
+ "content": "<|special_243|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "244": {
+ "content": "<|special_244|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "245": {
+ "content": "<|special_245|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "246": {
+ "content": "<|special_246|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "247": {
+ "content": "<|special_247|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "248": {
+ "content": "<|special_248|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "249": {
+ "content": "<|special_249|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "250": {
+ "content": "<|special_250|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "251": {
+ "content": "<|special_251|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "252": {
+ "content": "<|special_252|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "253": {
+ "content": "<|special_253|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "254": {
+ "content": "<|special_254|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "255": {
+ "content": "<|special_255|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "256": {
+ "content": "<|special_256|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "257": {
+ "content": "<|special_257|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "258": {
+ "content": "<|special_258|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "259": {
+ "content": "<|special_259|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "260": {
+ "content": "<|special_260|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "261": {
+ "content": "<|special_261|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "262": {
+ "content": "<|special_262|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "263": {
+ "content": "<|special_263|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "264": {
+ "content": "<|special_264|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "265": {
+ "content": "<|special_265|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "266": {
+ "content": "<|special_266|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "267": {
+ "content": "<|special_267|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "268": {
+ "content": "<|special_268|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "269": {
+ "content": "<|special_269|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "270": {
+ "content": "<|special_270|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "271": {
+ "content": "<|special_271|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "272": {
+ "content": "<|special_272|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "273": {
+ "content": "<|special_273|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "274": {
+ "content": "<|special_274|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "275": {
+ "content": "<|special_275|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "276": {
+ "content": "<|special_276|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "277": {
+ "content": "<|special_277|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "278": {
+ "content": "<|special_278|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "279": {
+ "content": "<|special_279|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "280": {
+ "content": "<|special_280|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "281": {
+ "content": "<|special_281|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "282": {
+ "content": "<|special_282|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "283": {
+ "content": "<|special_283|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "284": {
+ "content": "<|special_284|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "285": {
+ "content": "<|special_285|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "286": {
+ "content": "<|special_286|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "287": {
+ "content": "<|special_287|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "288": {
+ "content": "<|special_288|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "289": {
+ "content": "<|special_289|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "290": {
+ "content": "<|special_290|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "291": {
+ "content": "<|special_291|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "292": {
+ "content": "<|special_292|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "293": {
+ "content": "<|special_293|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "294": {
+ "content": "<|special_294|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "295": {
+ "content": "<|special_295|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "296": {
+ "content": "<|special_296|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "297": {
+ "content": "<|special_297|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "298": {
+ "content": "<|special_298|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "299": {
+ "content": "<|special_299|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "300": {
+ "content": "<|special_300|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "301": {
+ "content": "<|special_301|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "302": {
+ "content": "<|special_302|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "303": {
+ "content": "<|special_303|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "304": {
+ "content": "<|special_304|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "305": {
+ "content": "<|special_305|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "306": {
+ "content": "<|special_306|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "307": {
+ "content": "<|special_307|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "308": {
+ "content": "<|special_308|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "309": {
+ "content": "<|special_309|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "310": {
+ "content": "<|special_310|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "311": {
+ "content": "<|special_311|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "312": {
+ "content": "<|special_312|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "313": {
+ "content": "<|special_313|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "314": {
+ "content": "<|special_314|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "315": {
+ "content": "<|special_315|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "316": {
+ "content": "<|special_316|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "317": {
+ "content": "<|special_317|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "318": {
+ "content": "<|special_318|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "319": {
+ "content": "<|special_319|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "320": {
+ "content": "<|special_320|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "321": {
+ "content": "<|special_321|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "322": {
+ "content": "<|special_322|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "323": {
+ "content": "<|special_323|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "324": {
+ "content": "<|special_324|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "325": {
+ "content": "<|special_325|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "326": {
+ "content": "<|special_326|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "327": {
+ "content": "<|special_327|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "328": {
+ "content": "<|special_328|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "329": {
+ "content": "<|special_329|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "330": {
+ "content": "<|special_330|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "331": {
+ "content": "<|special_331|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "332": {
+ "content": "<|special_332|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "333": {
+ "content": "<|special_333|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "334": {
+ "content": "<|special_334|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "335": {
+ "content": "<|special_335|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "336": {
+ "content": "<|special_336|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "337": {
+ "content": "<|special_337|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "338": {
+ "content": "<|special_338|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "339": {
+ "content": "<|special_339|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "340": {
+ "content": "<|special_340|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "341": {
+ "content": "<|special_341|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "342": {
+ "content": "<|special_342|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "343": {
+ "content": "<|special_343|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "344": {
+ "content": "<|special_344|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "345": {
+ "content": "<|special_345|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "346": {
+ "content": "<|special_346|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "347": {
+ "content": "<|special_347|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "348": {
+ "content": "<|special_348|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "349": {
+ "content": "<|special_349|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "350": {
+ "content": "<|special_350|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "351": {
+ "content": "<|special_351|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "352": {
+ "content": "<|special_352|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "353": {
+ "content": "<|special_353|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "354": {
+ "content": "<|special_354|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "355": {
+ "content": "<|special_355|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "356": {
+ "content": "<|special_356|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "357": {
+ "content": "<|special_357|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "358": {
+ "content": "<|special_358|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "359": {
+ "content": "<|special_359|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "360": {
+ "content": "<|special_360|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "361": {
+ "content": "<|special_361|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "362": {
+ "content": "<|special_362|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "363": {
+ "content": "<|special_363|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "364": {
+ "content": "<|special_364|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "365": {
+ "content": "<|special_365|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "366": {
+ "content": "<|special_366|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "367": {
+ "content": "<|special_367|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "368": {
+ "content": "<|special_368|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "369": {
+ "content": "<|special_369|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "370": {
+ "content": "<|special_370|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "371": {
+ "content": "<|special_371|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "372": {
+ "content": "<|special_372|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "373": {
+ "content": "<|special_373|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "374": {
+ "content": "<|special_374|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "375": {
+ "content": "<|special_375|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "376": {
+ "content": "<|special_376|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "377": {
+ "content": "<|special_377|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "378": {
+ "content": "<|special_378|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "379": {
+ "content": "<|special_379|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "380": {
+ "content": "<|special_380|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "381": {
+ "content": "<|special_381|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "382": {
+ "content": "<|special_382|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "383": {
+ "content": "<|special_383|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "384": {
+ "content": "<|special_384|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "385": {
+ "content": "<|special_385|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "386": {
+ "content": "<|special_386|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "387": {
+ "content": "<|special_387|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "388": {
+ "content": "<|special_388|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "389": {
+ "content": "<|special_389|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "390": {
+ "content": "<|special_390|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "391": {
+ "content": "<|special_391|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "392": {
+ "content": "<|special_392|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "393": {
+ "content": "<|special_393|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "394": {
+ "content": "<|special_394|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "395": {
+ "content": "<|special_395|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "396": {
+ "content": "<|special_396|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "397": {
+ "content": "<|special_397|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "398": {
+ "content": "<|special_398|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "399": {
+ "content": "<|special_399|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "400": {
+ "content": "<|special_400|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "401": {
+ "content": "<|special_401|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "402": {
+ "content": "<|special_402|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "403": {
+ "content": "<|special_403|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "404": {
+ "content": "<|special_404|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "405": {
+ "content": "<|special_405|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "406": {
+ "content": "<|special_406|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "407": {
+ "content": "<|special_407|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "408": {
+ "content": "<|special_408|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "409": {
+ "content": "<|special_409|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "410": {
+ "content": "<|special_410|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "411": {
+ "content": "<|special_411|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "412": {
+ "content": "<|special_412|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "413": {
+ "content": "<|special_413|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "414": {
+ "content": "<|special_414|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "415": {
+ "content": "<|special_415|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "416": {
+ "content": "<|special_416|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "417": {
+ "content": "<|special_417|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "418": {
+ "content": "<|special_418|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "419": {
+ "content": "<|special_419|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "420": {
+ "content": "<|special_420|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "421": {
+ "content": "<|special_421|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "422": {
+ "content": "<|special_422|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "423": {
+ "content": "<|special_423|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "424": {
+ "content": "<|special_424|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "425": {
+ "content": "<|special_425|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "426": {
+ "content": "<|special_426|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "427": {
+ "content": "<|special_427|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "428": {
+ "content": "<|special_428|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "429": {
+ "content": "<|special_429|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "430": {
+ "content": "<|special_430|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "431": {
+ "content": "<|special_431|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "432": {
+ "content": "<|special_432|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "433": {
+ "content": "<|special_433|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "434": {
+ "content": "<|special_434|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "435": {
+ "content": "<|special_435|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "436": {
+ "content": "<|special_436|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "437": {
+ "content": "<|special_437|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "438": {
+ "content": "<|special_438|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "439": {
+ "content": "<|special_439|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "440": {
+ "content": "<|special_440|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "441": {
+ "content": "<|special_441|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "442": {
+ "content": "<|special_442|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "443": {
+ "content": "<|special_443|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "444": {
+ "content": "<|special_444|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "445": {
+ "content": "<|special_445|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "446": {
+ "content": "<|special_446|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "447": {
+ "content": "<|special_447|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "448": {
+ "content": "<|special_448|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "449": {
+ "content": "<|special_449|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "450": {
+ "content": "<|special_450|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "451": {
+ "content": "<|special_451|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "452": {
+ "content": "<|special_452|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "453": {
+ "content": "<|special_453|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "454": {
+ "content": "<|special_454|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "455": {
+ "content": "<|special_455|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "456": {
+ "content": "<|special_456|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "457": {
+ "content": "<|special_457|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "458": {
+ "content": "<|special_458|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "459": {
+ "content": "<|special_459|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "460": {
+ "content": "<|special_460|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "461": {
+ "content": "<|special_461|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "462": {
+ "content": "<|special_462|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "463": {
+ "content": "<|special_463|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "464": {
+ "content": "<|special_464|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "465": {
+ "content": "<|special_465|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "466": {
+ "content": "<|special_466|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "467": {
+ "content": "<|special_467|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "468": {
+ "content": "<|special_468|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "469": {
+ "content": "<|special_469|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "470": {
+ "content": "<|special_470|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "471": {
+ "content": "<|special_471|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "472": {
+ "content": "<|special_472|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "473": {
+ "content": "<|special_473|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "474": {
+ "content": "<|special_474|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "475": {
+ "content": "<|special_475|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "476": {
+ "content": "<|special_476|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "477": {
+ "content": "<|special_477|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "478": {
+ "content": "<|special_478|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "479": {
+ "content": "<|special_479|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "480": {
+ "content": "<|special_480|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "481": {
+ "content": "<|special_481|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "482": {
+ "content": "<|special_482|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "483": {
+ "content": "<|special_483|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "484": {
+ "content": "<|special_484|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "485": {
+ "content": "<|special_485|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "486": {
+ "content": "<|special_486|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "487": {
+ "content": "<|special_487|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "488": {
+ "content": "<|special_488|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "489": {
+ "content": "<|special_489|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "490": {
+ "content": "<|special_490|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "491": {
+ "content": "<|special_491|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "492": {
+ "content": "<|special_492|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "493": {
+ "content": "<|special_493|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "494": {
+ "content": "<|special_494|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "495": {
+ "content": "<|special_495|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "496": {
+ "content": "<|special_496|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "497": {
+ "content": "<|special_497|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "498": {
+ "content": "<|special_498|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "499": {
+ "content": "<|special_499|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "500": {
+ "content": "<|special_500|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "501": {
+ "content": "<|special_501|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "502": {
+ "content": "<|special_502|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "503": {
+ "content": "<|special_503|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "504": {
+ "content": "<|special_504|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "505": {
+ "content": "<|special_505|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "506": {
+ "content": "<|special_506|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "507": {
+ "content": "<|special_507|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "508": {
+ "content": "<|special_508|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "509": {
+ "content": "<|special_509|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "510": {
+ "content": "<|special_510|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "511": {
+ "content": "<|special_511|>",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": true
+ },
+ "512": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "513": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "514": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "515": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "516": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "517": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "518": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "519": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "520": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "521": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "522": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "523": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "524": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "525": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "526": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "527": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "528": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "529": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "530": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "531": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "532": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "533": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "534": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "535": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "536": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "537": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "538": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "539": {
+ "content": "",
+ "lstrip": false,
+ "normalized": false,
+ "rstrip": false,
+ "single_word": false,
+ "special": false
+ },
+ "540": {
+ "content": "