feat: publish Fish Audio S2 Pro MLX quantized

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE.md

@@ -0,0 +1,94 @@
+# FISH AUDIO RESEARCH LICENSE AGREEMENT
+
+**Last Updated: March 7, 2026**
+
+## I. INTRODUCTION
+
+This Agreement applies to any individual person or entity ("You", "Your" or "Licensee") that uses or distributes any portion or element of the Fish Audio Materials or Derivative Works thereof for any Research, Non-Commercial, or Commercial purpose. Capitalized terms not otherwise defined herein are defined in Section V below.
+
+This Agreement is intended to allow research and non-commercial uses of the Materials free of charge. Any Commercial use of the Materials requires a separate license from Fish Audio.
+
+By clicking "I Accept" or by using, distributing, or accessing any portion or element of the Fish Audio Materials or Derivative Works, You agree that You have read, understood and are bound by the terms of this Agreement. If You are acting on behalf of a company, organization or other entity, then "You" includes you and that entity, and You agree that You: (i) are an authorized representative of such entity with the authority to bind such entity to this Agreement, and (ii) You agree to the terms of this Agreement on that entity's behalf.
+
+## II. RESEARCH & NON-COMMERCIAL USE LICENSE
+
+Subject to the terms of this Agreement, Fish Audio grants You a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable and royalty-free limited license under Fish Audio's intellectual property or other rights owned by Fish Audio embodied in the Fish Audio Materials to use, reproduce, distribute, and create Derivative Works of, and make modifications to, the Fish Audio Materials for any Research or Non-Commercial Purpose.
+
+"Research Purpose" means academic or scientific advancement, and in each case, is not primarily intended for commercial advantage or monetary compensation to You or others.
+
+"Non-Commercial Purpose" means any purpose other than a Research Purpose that is not primarily intended for commercial advantage or monetary compensation to You or others, such as personal use (i.e., hobbyist) or evaluation and testing.
+
+## III. COMMERCIAL USE
+
+**Any use of the Fish Audio Materials or Derivative Works for a Commercial Purpose requires a separate written license agreement from Fish Audio.** No commercial rights are granted under this Agreement.
+
+"Commercial Purpose" means any purpose other than a Research Purpose or Non-Commercial Purpose that is primarily intended for or directed toward commercial advantage or monetary compensation to You or others, including but not limited to: (i) creating, modifying, or distributing Your product or service, including via a hosted service or application programming interface, (ii) Your business's or organization's internal operations, and (iii) any use in connection with a product or service for which You charge a fee or generate revenue, whether directly or indirectly.
+
+To obtain a commercial license, please contact Fish Audio at:
+
+- **Website:** [https://fish.audio](https://fish.audio)
+- **Email:** business@fish.audio
+
+## IV. GENERAL TERMS
+
+Your Research and Non-Commercial License under this Agreement is subject to the following terms.
+
+### a. Distribution & Attribution
+
+If You distribute or make available the Fish Audio Materials or a Derivative Work to a third party, or a product or service that uses any portion of them, You shall: (i) provide a copy of this Agreement to that third party, (ii) retain the following attribution notice within a "Notice" text file distributed as a part of such copies: "This model is licensed under the Fish Audio Research License, Copyright © 39 AI, INC. All Rights Reserved.", and (iii) prominently display "Built with Fish Audio" on a related website, user interface, blogpost, about page, or product documentation.
+
+If You create a Derivative Work, You may add your own attribution notice(s) to the "Notice" text file included with that Derivative Work, provided that You clearly indicate which attributions apply to the Fish Audio Materials and state in the "Notice" text file that You changed the Fish Audio Materials and how it was modified.
+
+### b. Use Restrictions
+
+Your use of the Fish Audio Materials and Derivative Works, including any output or results of the Fish Audio Materials or Derivative Works, must comply with applicable laws and regulations (including Trade Control Laws and equivalent regulations) and adhere to Fish Audio's Acceptable Use Policy, which is hereby incorporated by reference.
+
+Furthermore, You will not use the Fish Audio Materials or Derivative Works, or any output or results of the Fish Audio Materials or Derivative Works, to create or improve any foundational generative AI model (excluding the Models or Derivative Works).
+
+### c. Intellectual Property
+
+**(i) Trademark License.** No trademark licenses are granted under this Agreement, and in connection with the Fish Audio Materials or Derivative Works, You may not use any name or mark owned by or associated with Fish Audio or any of its Affiliates, except as required under Section IV(a) herein.
+
+**(ii) Ownership of Derivative Works.** As between You and Fish Audio, You are the owner of Derivative Works You create, subject to Fish Audio's ownership of the Fish Audio Materials and any Derivative Works made by or for Fish Audio.
+
+**(iii) Ownership of Outputs.** As between You and Fish Audio, You own any outputs generated from the Models or Derivative Works to the extent permitted by applicable law.
+
+**(iv) Disputes.** If You or Your Affiliate(s) institute litigation or other proceedings against Fish Audio (including a cross-claim or counterclaim in a lawsuit) alleging that the Fish Audio Materials, Derivative Works or associated outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by You, then any licenses granted to You under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Fish Audio from and against any claim by any third party arising out of or related to Your use or distribution of the Fish Audio Materials or Derivative Works in violation of this Agreement.
+
+**(v) Feedback.** From time to time, You may provide Fish Audio with verbal and/or written suggestions, comments or other feedback related to Fish Audio's existing or prospective technology, products or services (collectively, "Feedback"). You are not obligated to provide Fish Audio with Feedback, but to the extent that You do, You hereby grant Fish Audio a perpetual, irrevocable, royalty-free, fully-paid, sub-licensable, transferable, non-exclusive, worldwide right and license to exploit the Feedback in any manner without restriction. Your Feedback is provided "AS IS" and You make no warranties whatsoever about any Feedback.
+
+### d. Disclaimer of Warranty
+
+UNLESS REQUIRED BY APPLICABLE LAW, THE FISH AUDIO MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OR LAWFULNESS OF USING OR REDISTRIBUTING THE FISH AUDIO MATERIALS, DERIVATIVE WORKS OR ANY OUTPUT OR RESULTS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE FISH AUDIO MATERIALS, DERIVATIVE WORKS AND ANY OUTPUT AND RESULTS.
+
+### e. Limitation of Liability
+
+IN NO EVENT WILL FISH AUDIO OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF FISH AUDIO OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+### f. Term and Termination
+
+The term of this Agreement will commence upon Your acceptance of this Agreement or access to the Fish Audio Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Fish Audio may terminate this Agreement if You are in breach of any term or condition of this Agreement. Upon termination of this Agreement, You shall delete and cease use of any Fish Audio Materials or Derivative Works. Sections IV(d), (e), and (g) shall survive the termination of this Agreement.
+
+### g. Governing Law
+
+This Agreement will be governed by and construed in accordance with the laws of the United States and the State of California without regard to choice of law principles, and the UN Convention on Contracts for International Sale of Goods does not apply to this Agreement.
+
+## V. DEFINITIONS
+
+**"Affiliate(s)"** means any entity that directly or indirectly controls, is controlled by, or is under common control with the subject entity; for purposes of this definition, "control" means direct or indirect ownership or control of more than 50% of the voting interests of the subject entity.
+
+**"Agreement"** means this Fish Audio Research License Agreement.
+
+**"Derivative Work(s)"** means (a) any derivative work of the Fish Audio Materials as recognized by U.S. copyright laws and (b) any modifications to a Model, and any other model created which is based on or derived from the Model or the Model's output, including "fine tune" and "low-rank adaptation" models derived from a Model or a Model's output, but do not include the output of any Model.
+
+**"Documentation"** means any specifications, manuals, documentation, and other written information provided by Fish Audio related to the Software or Models.
+
+**"Fish Audio"** or **"we"** means 39 AI, INC. and its Affiliates.
+
+**"Model(s)"** means, collectively, Fish Audio's proprietary models and algorithms, including machine-learning models, trained model weights and other elements of the foregoing.
+
+**"Software"** means Fish Audio's proprietary software made available under this Agreement now or in the future.
+
+**"Fish Audio Materials"** means, collectively, Fish Audio's proprietary Models, Software and Documentation (and any portion or combination thereof) made available under this Agreement.
+
+**"Trade Control Laws"** means any applicable U.S. and non-U.S. export control and trade sanctions laws and regulations.

NOTICE

@@ -0,0 +1,11 @@
+This model is licensed under the Fish Audio Research License.
+Copyright © 39 AI, INC. All Rights Reserved.
+
+Modifications to the Fish Audio Materials in this derivative work:
+- Conversion to Apple MLX format via mlx-audio (bf16 → quantized int weights)
+- Saved as model.safetensors with mlx-specific layout and metadata
+- No changes to model architecture, tokenizer, codec, or training data
+
+Built with Fish Audio.
+
+See LICENSE.md for the full license terms.

README.md

@@ -0,0 +1,90 @@
+---
+library_name: mlx-audio
+tags:
+- mlx
+- mlx-audio
+- text-to-speech
+- speech
+- speech-generation
+- voice-cloning
+- tts
+- fish-audio
+- s2-pro
+- quantized
+- 8bit
+- fish_qwen3_omni
+base_model: fishaudio/s2-pro
+license: other
+license_name: fish-audio-research
+license_link: https://huggingface.co/fishaudio/s2-pro/blob/main/LICENSE
+pipeline_tag: text-to-speech
+language:
+- en
+- zh
+- ja
+- ko
+- es
+- pt
+- ar
+- ru
+- fr
+- de
+---
+
+# Fish Audio S2 Pro MLX 8-bit
+
+MLX 8-bit quantization of [fishaudio/s2-pro](https://huggingface.co/fishaudio/s2-pro), produced with [mlx-audio](https://github.com/Blaizzy/mlx-audio) on Apple Silicon.
+
+**Built with Fish Audio.**
+
+## Provenance
+
+Converted directly from `fishaudio/s2-pro` bf16 safetensors via `python -m mlx_audio.convert ... -q --q-bits 4 --q-group-size 64`. The `codec.pth` audio decoder, tokenizer, and chat template are preserved alongside the quantized LLM weights.
+
+## Quantization
+
+- Method: MLX affine quantization, group_size=64, 8-bit
+- Actual bits per weight after conversion: **~9.0** (some small projections are skipped and remain bf16 — this is expected behavior for mlx.nn.quantize on multimodal models)
+- Size on disk: ~4.5 GB (vs ~8 GB for bf16)
+- Target: Apple Silicon with ≥ 16 GB unified memory (comfortable on M-series with 24 GB+)
+
+## Architecture
+
+`fish_qwen3_omni` — a Qwen3-based decoder-only LLM that emits audio tokens consumed by the `codec.pth` DualAR codec. The text side is quantized; the codec is kept as-is.
+
+## Quickstart
+
+```python
+from mlx_audio.tts import load
+
+model = load("majentik/fishaudio-s2-pro-MLX-8bit")
+audio = model.generate(
+    text="Hello, this is a test of Fish S2 Pro at 8-bit MLX.",
+    voice="en_default",
+)
+```
+
+Fish S2 Pro supports zero-shot voice cloning from a reference audio sample; see the base model card for the reference-conditioning API.
+
+## License & Use Restrictions
+
+This model is a derivative work of **Fish Audio S2 Pro**, licensed under the **Fish Audio Research License**.
+
+- **Research + Non-Commercial use:** free, subject to the license.
+- **Commercial use:** requires a separate written agreement with Fish Audio — see https://fish.audio or email `business@fish.audio`.
+- **Use restriction:** outputs must not be used to create or improve any foundational generative AI model (excluding the Model itself or derivatives of it).
+- **Attribution:** downstream use must include this NOTICE and display "Built with Fish Audio" in any product/service/interface that surfaces model outputs.
+
+See `LICENSE.md` and `NOTICE` files bundled with this repository.
+
+## Languages
+
+English, Chinese, Japanese, Korean, Spanish, Portuguese, Arabic, Russian, French, German (per base model card).
+
+## See also
+
+- Base: [fishaudio/s2-pro](https://huggingface.co/fishaudio/s2-pro)
+- bf16 MLX: [mlx-community/fish-audio-s2-pro-bf16](https://huggingface.co/mlx-community/fish-audio-s2-pro-bf16)
+- 8-bit MLX: [mlx-community/fishaudio-s2-pro-8bit-mlx](https://huggingface.co/mlx-community/fishaudio-s2-pro-8bit-mlx)
+- mlx-audio: https://github.com/Blaizzy/mlx-audio
+- Garden hub: [majentik/garden](https://huggingface.co/majentik/garden)

chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

codec.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74fc41c5a7151c6f350af8bd7e5d6e3accfcc7f3dfbfac23afd35af07052bb2f
+size 1871099728

config.json

@@ -0,0 +1,80 @@
+{
+    "audio_decoder_config": {
+        "attention_o_bias": false,
+        "attention_qk_norm": false,
+        "attention_qkv_bias": false,
+        "audio_hidden_dim": 5120,
+        "dim": 2560,
+        "dropout": 0.0,
+        "head_dim": 128,
+        "initializer_range": 0.01976423537605237,
+        "intermediate_size": 9728,
+        "max_seq_len": 11,
+        "model_type": "fish_qwen3_audio_decoder",
+        "moe_intermediate_size": 768,
+        "n_head": 32,
+        "n_layer": 4,
+        "n_local_heads": 8,
+        "norm_eps": 1e-06,
+        "norm_topk_prob": true,
+        "num_codebooks": 10,
+        "num_experts": 1,
+        "num_experts_per_tok": 1,
+        "rope_base": 1000000,
+        "router_gamma": 0.001,
+        "text_dim": 2560,
+        "tie_word_embeddings": false,
+        "use_aux_loss_free": false,
+        "use_bfloat16": false,
+        "use_gradient_checkpointing": true,
+        "use_moe": false,
+        "vocab_size": 4096
+    },
+    "audio_pad_token_id": 151677,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "model_type": "fish_qwen3_omni",
+    "pad_token_id": 151669,
+    "quantization": {
+        "group_size": 64,
+        "bits": 8,
+        "mode": "affine"
+    },
+    "quantization_config": {
+        "group_size": 64,
+        "bits": 8,
+        "mode": "affine"
+    },
+    "semantic_end_token_id": 155773,
+    "semantic_start_token_id": 151678,
+    "text_config": {
+        "attention_o_bias": false,
+        "attention_qk_norm": true,
+        "attention_qkv_bias": false,
+        "audio_hidden_dim": 5120,
+        "dim": 2560,
+        "dropout": 0.0,
+        "head_dim": 128,
+        "initializer_range": 0.01976423537605237,
+        "intermediate_size": 9728,
+        "max_seq_len": 32768,
+        "model_type": "fish_qwen3",
+        "moe_intermediate_size": 768,
+        "n_head": 32,
+        "n_layer": 36,
+        "n_local_heads": 8,
+        "norm_eps": 1e-06,
+        "norm_topk_prob": true,
+        "num_experts": 1,
+        "num_experts_per_tok": 1,
+        "rope_base": 1000000,
+        "router_gamma": 0.001,
+        "tie_word_embeddings": true,
+        "use_aux_loss_free": false,
+        "use_bfloat16": false,
+        "use_gradient_checkpointing": true,
+        "use_moe": false,
+        "vocab_size": 155776
+    },
+    "transformers_version": "4.57.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf31b4e5fb713a565750e120ebadd54de245eb1214e46d14fc5108f1e79171a1
+size 4847260185

model.safetensors.index.json

@@ -0,0 +1,774 @@
+{
+    "metadata": {
+        "total_size": 4847173632,
+        "total_parameters": 4561852416
+    },
+    "weight_map": {
+        "model.codebook_embeddings.biases": "model.safetensors",
+        "model.codebook_embeddings.scales": "model.safetensors",
+        "model.codebook_embeddings.weight": "model.safetensors",
+        "model.embeddings.biases": "model.safetensors",
+        "model.embeddings.scales": "model.safetensors",
+        "model.embeddings.weight": "model.safetensors",
+        "model.fast_embeddings.biases": "model.safetensors",
+        "model.fast_embeddings.scales": "model.safetensors",
+        "model.fast_embeddings.weight": "model.safetensors",
+        "model.fast_layers.0.attention.wo.biases": "model.safetensors",
+        "model.fast_layers.0.attention.wo.scales": "model.safetensors",
+        "model.fast_layers.0.attention.wo.weight": "model.safetensors",
+        "model.fast_layers.0.attention.wqkv.biases": "model.safetensors",
+        "model.fast_layers.0.attention.wqkv.scales": "model.safetensors",
+        "model.fast_layers.0.attention.wqkv.weight": "model.safetensors",
+        "model.fast_layers.0.attention_norm.weight": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w1.biases": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w1.scales": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w1.weight": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w2.biases": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w2.scales": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w2.weight": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w3.biases": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w3.scales": "model.safetensors",
+        "model.fast_layers.0.feed_forward.w3.weight": "model.safetensors",
+        "model.fast_layers.0.ffn_norm.weight": "model.safetensors",
+        "model.fast_layers.1.attention.wo.biases": "model.safetensors",
+        "model.fast_layers.1.attention.wo.scales": "model.safetensors",
+        "model.fast_layers.1.attention.wo.weight": "model.safetensors",
+        "model.fast_layers.1.attention.wqkv.biases": "model.safetensors",
+        "model.fast_layers.1.attention.wqkv.scales": "model.safetensors",
+        "model.fast_layers.1.attention.wqkv.weight": "model.safetensors",
+        "model.fast_layers.1.attention_norm.weight": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w1.biases": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w1.scales": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w1.weight": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w2.biases": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w2.scales": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w2.weight": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w3.biases": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w3.scales": "model.safetensors",
+        "model.fast_layers.1.feed_forward.w3.weight": "model.safetensors",
+        "model.fast_layers.1.ffn_norm.weight": "model.safetensors",
+        "model.fast_layers.2.attention.wo.biases": "model.safetensors",
+        "model.fast_layers.2.attention.wo.scales": "model.safetensors",
+        "model.fast_layers.2.attention.wo.weight": "model.safetensors",
+        "model.fast_layers.2.attention.wqkv.biases": "model.safetensors",
+        "model.fast_layers.2.attention.wqkv.scales": "model.safetensors",
+        "model.fast_layers.2.attention.wqkv.weight": "model.safetensors",
+        "model.fast_layers.2.attention_norm.weight": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w1.biases": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w1.scales": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w1.weight": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w2.biases": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w2.scales": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w2.weight": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w3.biases": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w3.scales": "model.safetensors",
+        "model.fast_layers.2.feed_forward.w3.weight": "model.safetensors",
+        "model.fast_layers.2.ffn_norm.weight": "model.safetensors",
+        "model.fast_layers.3.attention.wo.biases": "model.safetensors",
+        "model.fast_layers.3.attention.wo.scales": "model.safetensors",
+        "model.fast_layers.3.attention.wo.weight": "model.safetensors",
+        "model.fast_layers.3.attention.wqkv.biases": "model.safetensors",
+        "model.fast_layers.3.attention.wqkv.scales": "model.safetensors",
+        "model.fast_layers.3.attention.wqkv.weight": "model.safetensors",
+        "model.fast_layers.3.attention_norm.weight": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w1.biases": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w1.scales": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w1.weight": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w2.biases": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w2.scales": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w2.weight": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w3.biases": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w3.scales": "model.safetensors",
+        "model.fast_layers.3.feed_forward.w3.weight": "model.safetensors",
+        "model.fast_layers.3.ffn_norm.weight": "model.safetensors",
+        "model.fast_norm.weight": "model.safetensors",
+        "model.fast_output.biases": "model.safetensors",
+        "model.fast_output.scales": "model.safetensors",
+        "model.fast_output.weight": "model.safetensors",
+        "model.layers.0.attention.k_norm.weight": "model.safetensors",
+        "model.layers.0.attention.q_norm.weight": "model.safetensors",
+        "model.layers.0.attention.wo.biases": "model.safetensors",
+        "model.layers.0.attention.wo.scales": "model.safetensors",
+        "model.layers.0.attention.wo.weight": "model.safetensors",
+        "model.layers.0.attention.wqkv.biases": "model.safetensors",
+        "model.layers.0.attention.wqkv.scales": "model.safetensors",
+        "model.layers.0.attention.wqkv.weight": "model.safetensors",
+        "model.layers.0.attention_norm.weight": "model.safetensors",
+        "model.layers.0.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.0.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.0.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.0.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.0.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.0.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.0.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.0.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.0.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.0.ffn_norm.weight": "model.safetensors",
+        "model.layers.1.attention.k_norm.weight": "model.safetensors",
+        "model.layers.1.attention.q_norm.weight": "model.safetensors",
+        "model.layers.1.attention.wo.biases": "model.safetensors",
+        "model.layers.1.attention.wo.scales": "model.safetensors",
+        "model.layers.1.attention.wo.weight": "model.safetensors",
+        "model.layers.1.attention.wqkv.biases": "model.safetensors",
+        "model.layers.1.attention.wqkv.scales": "model.safetensors",
+        "model.layers.1.attention.wqkv.weight": "model.safetensors",
+        "model.layers.1.attention_norm.weight": "model.safetensors",
+        "model.layers.1.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.1.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.1.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.1.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.1.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.1.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.1.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.1.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.1.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.1.ffn_norm.weight": "model.safetensors",
+        "model.layers.10.attention.k_norm.weight": "model.safetensors",
+        "model.layers.10.attention.q_norm.weight": "model.safetensors",
+        "model.layers.10.attention.wo.biases": "model.safetensors",
+        "model.layers.10.attention.wo.scales": "model.safetensors",
+        "model.layers.10.attention.wo.weight": "model.safetensors",
+        "model.layers.10.attention.wqkv.biases": "model.safetensors",
+        "model.layers.10.attention.wqkv.scales": "model.safetensors",
+        "model.layers.10.attention.wqkv.weight": "model.safetensors",
+        "model.layers.10.attention_norm.weight": "model.safetensors",
+        "model.layers.10.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.10.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.10.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.10.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.10.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.10.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.10.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.10.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.10.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.10.ffn_norm.weight": "model.safetensors",
+        "model.layers.11.attention.k_norm.weight": "model.safetensors",
+        "model.layers.11.attention.q_norm.weight": "model.safetensors",
+        "model.layers.11.attention.wo.biases": "model.safetensors",
+        "model.layers.11.attention.wo.scales": "model.safetensors",
+        "model.layers.11.attention.wo.weight": "model.safetensors",
+        "model.layers.11.attention.wqkv.biases": "model.safetensors",
+        "model.layers.11.attention.wqkv.scales": "model.safetensors",
+        "model.layers.11.attention.wqkv.weight": "model.safetensors",
+        "model.layers.11.attention_norm.weight": "model.safetensors",
+        "model.layers.11.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.11.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.11.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.11.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.11.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.11.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.11.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.11.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.11.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.11.ffn_norm.weight": "model.safetensors",
+        "model.layers.12.attention.k_norm.weight": "model.safetensors",
+        "model.layers.12.attention.q_norm.weight": "model.safetensors",
+        "model.layers.12.attention.wo.biases": "model.safetensors",
+        "model.layers.12.attention.wo.scales": "model.safetensors",
+        "model.layers.12.attention.wo.weight": "model.safetensors",
+        "model.layers.12.attention.wqkv.biases": "model.safetensors",
+        "model.layers.12.attention.wqkv.scales": "model.safetensors",
+        "model.layers.12.attention.wqkv.weight": "model.safetensors",
+        "model.layers.12.attention_norm.weight": "model.safetensors",
+        "model.layers.12.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.12.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.12.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.12.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.12.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.12.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.12.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.12.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.12.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.12.ffn_norm.weight": "model.safetensors",
+        "model.layers.13.attention.k_norm.weight": "model.safetensors",
+        "model.layers.13.attention.q_norm.weight": "model.safetensors",
+        "model.layers.13.attention.wo.biases": "model.safetensors",
+        "model.layers.13.attention.wo.scales": "model.safetensors",
+        "model.layers.13.attention.wo.weight": "model.safetensors",
+        "model.layers.13.attention.wqkv.biases": "model.safetensors",
+        "model.layers.13.attention.wqkv.scales": "model.safetensors",
+        "model.layers.13.attention.wqkv.weight": "model.safetensors",
+        "model.layers.13.attention_norm.weight": "model.safetensors",
+        "model.layers.13.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.13.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.13.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.13.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.13.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.13.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.13.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.13.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.13.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.13.ffn_norm.weight": "model.safetensors",
+        "model.layers.14.attention.k_norm.weight": "model.safetensors",
+        "model.layers.14.attention.q_norm.weight": "model.safetensors",
+        "model.layers.14.attention.wo.biases": "model.safetensors",
+        "model.layers.14.attention.wo.scales": "model.safetensors",
+        "model.layers.14.attention.wo.weight": "model.safetensors",
+        "model.layers.14.attention.wqkv.biases": "model.safetensors",
+        "model.layers.14.attention.wqkv.scales": "model.safetensors",
+        "model.layers.14.attention.wqkv.weight": "model.safetensors",
+        "model.layers.14.attention_norm.weight": "model.safetensors",
+        "model.layers.14.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.14.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.14.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.14.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.14.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.14.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.14.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.14.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.14.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.14.ffn_norm.weight": "model.safetensors",
+        "model.layers.15.attention.k_norm.weight": "model.safetensors",
+        "model.layers.15.attention.q_norm.weight": "model.safetensors",
+        "model.layers.15.attention.wo.biases": "model.safetensors",
+        "model.layers.15.attention.wo.scales": "model.safetensors",
+        "model.layers.15.attention.wo.weight": "model.safetensors",
+        "model.layers.15.attention.wqkv.biases": "model.safetensors",
+        "model.layers.15.attention.wqkv.scales": "model.safetensors",
+        "model.layers.15.attention.wqkv.weight": "model.safetensors",
+        "model.layers.15.attention_norm.weight": "model.safetensors",
+        "model.layers.15.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.15.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.15.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.15.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.15.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.15.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.15.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.15.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.15.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.15.ffn_norm.weight": "model.safetensors",
+        "model.layers.16.attention.k_norm.weight": "model.safetensors",
+        "model.layers.16.attention.q_norm.weight": "model.safetensors",
+        "model.layers.16.attention.wo.biases": "model.safetensors",
+        "model.layers.16.attention.wo.scales": "model.safetensors",
+        "model.layers.16.attention.wo.weight": "model.safetensors",
+        "model.layers.16.attention.wqkv.biases": "model.safetensors",
+        "model.layers.16.attention.wqkv.scales": "model.safetensors",
+        "model.layers.16.attention.wqkv.weight": "model.safetensors",
+        "model.layers.16.attention_norm.weight": "model.safetensors",
+        "model.layers.16.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.16.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.16.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.16.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.16.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.16.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.16.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.16.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.16.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.16.ffn_norm.weight": "model.safetensors",
+        "model.layers.17.attention.k_norm.weight": "model.safetensors",
+        "model.layers.17.attention.q_norm.weight": "model.safetensors",
+        "model.layers.17.attention.wo.biases": "model.safetensors",
+        "model.layers.17.attention.wo.scales": "model.safetensors",
+        "model.layers.17.attention.wo.weight": "model.safetensors",
+        "model.layers.17.attention.wqkv.biases": "model.safetensors",
+        "model.layers.17.attention.wqkv.scales": "model.safetensors",
+        "model.layers.17.attention.wqkv.weight": "model.safetensors",
+        "model.layers.17.attention_norm.weight": "model.safetensors",
+        "model.layers.17.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.17.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.17.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.17.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.17.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.17.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.17.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.17.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.17.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.17.ffn_norm.weight": "model.safetensors",
+        "model.layers.18.attention.k_norm.weight": "model.safetensors",
+        "model.layers.18.attention.q_norm.weight": "model.safetensors",
+        "model.layers.18.attention.wo.biases": "model.safetensors",
+        "model.layers.18.attention.wo.scales": "model.safetensors",
+        "model.layers.18.attention.wo.weight": "model.safetensors",
+        "model.layers.18.attention.wqkv.biases": "model.safetensors",
+        "model.layers.18.attention.wqkv.scales": "model.safetensors",
+        "model.layers.18.attention.wqkv.weight": "model.safetensors",
+        "model.layers.18.attention_norm.weight": "model.safetensors",
+        "model.layers.18.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.18.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.18.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.18.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.18.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.18.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.18.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.18.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.18.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.18.ffn_norm.weight": "model.safetensors",
+        "model.layers.19.attention.k_norm.weight": "model.safetensors",
+        "model.layers.19.attention.q_norm.weight": "model.safetensors",
+        "model.layers.19.attention.wo.biases": "model.safetensors",
+        "model.layers.19.attention.wo.scales": "model.safetensors",
+        "model.layers.19.attention.wo.weight": "model.safetensors",
+        "model.layers.19.attention.wqkv.biases": "model.safetensors",
+        "model.layers.19.attention.wqkv.scales": "model.safetensors",
+        "model.layers.19.attention.wqkv.weight": "model.safetensors",
+        "model.layers.19.attention_norm.weight": "model.safetensors",
+        "model.layers.19.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.19.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.19.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.19.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.19.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.19.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.19.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.19.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.19.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.19.ffn_norm.weight": "model.safetensors",
+        "model.layers.2.attention.k_norm.weight": "model.safetensors",
+        "model.layers.2.attention.q_norm.weight": "model.safetensors",
+        "model.layers.2.attention.wo.biases": "model.safetensors",
+        "model.layers.2.attention.wo.scales": "model.safetensors",
+        "model.layers.2.attention.wo.weight": "model.safetensors",
+        "model.layers.2.attention.wqkv.biases": "model.safetensors",
+        "model.layers.2.attention.wqkv.scales": "model.safetensors",
+        "model.layers.2.attention.wqkv.weight": "model.safetensors",
+        "model.layers.2.attention_norm.weight": "model.safetensors",
+        "model.layers.2.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.2.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.2.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.2.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.2.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.2.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.2.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.2.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.2.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.2.ffn_norm.weight": "model.safetensors",
+        "model.layers.20.attention.k_norm.weight": "model.safetensors",
+        "model.layers.20.attention.q_norm.weight": "model.safetensors",
+        "model.layers.20.attention.wo.biases": "model.safetensors",
+        "model.layers.20.attention.wo.scales": "model.safetensors",
+        "model.layers.20.attention.wo.weight": "model.safetensors",
+        "model.layers.20.attention.wqkv.biases": "model.safetensors",
+        "model.layers.20.attention.wqkv.scales": "model.safetensors",
+        "model.layers.20.attention.wqkv.weight": "model.safetensors",
+        "model.layers.20.attention_norm.weight": "model.safetensors",
+        "model.layers.20.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.20.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.20.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.20.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.20.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.20.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.20.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.20.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.20.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.20.ffn_norm.weight": "model.safetensors",
+        "model.layers.21.attention.k_norm.weight": "model.safetensors",
+        "model.layers.21.attention.q_norm.weight": "model.safetensors",
+        "model.layers.21.attention.wo.biases": "model.safetensors",
+        "model.layers.21.attention.wo.scales": "model.safetensors",
+        "model.layers.21.attention.wo.weight": "model.safetensors",
+        "model.layers.21.attention.wqkv.biases": "model.safetensors",
+        "model.layers.21.attention.wqkv.scales": "model.safetensors",
+        "model.layers.21.attention.wqkv.weight": "model.safetensors",
+        "model.layers.21.attention_norm.weight": "model.safetensors",
+        "model.layers.21.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.21.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.21.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.21.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.21.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.21.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.21.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.21.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.21.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.21.ffn_norm.weight": "model.safetensors",
+        "model.layers.22.attention.k_norm.weight": "model.safetensors",
+        "model.layers.22.attention.q_norm.weight": "model.safetensors",
+        "model.layers.22.attention.wo.biases": "model.safetensors",
+        "model.layers.22.attention.wo.scales": "model.safetensors",
+        "model.layers.22.attention.wo.weight": "model.safetensors",
+        "model.layers.22.attention.wqkv.biases": "model.safetensors",
+        "model.layers.22.attention.wqkv.scales": "model.safetensors",
+        "model.layers.22.attention.wqkv.weight": "model.safetensors",
+        "model.layers.22.attention_norm.weight": "model.safetensors",
+        "model.layers.22.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.22.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.22.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.22.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.22.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.22.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.22.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.22.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.22.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.22.ffn_norm.weight": "model.safetensors",
+        "model.layers.23.attention.k_norm.weight": "model.safetensors",
+        "model.layers.23.attention.q_norm.weight": "model.safetensors",
+        "model.layers.23.attention.wo.biases": "model.safetensors",
+        "model.layers.23.attention.wo.scales": "model.safetensors",
+        "model.layers.23.attention.wo.weight": "model.safetensors",
+        "model.layers.23.attention.wqkv.biases": "model.safetensors",
+        "model.layers.23.attention.wqkv.scales": "model.safetensors",
+        "model.layers.23.attention.wqkv.weight": "model.safetensors",
+        "model.layers.23.attention_norm.weight": "model.safetensors",
+        "model.layers.23.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.23.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.23.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.23.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.23.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.23.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.23.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.23.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.23.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.23.ffn_norm.weight": "model.safetensors",
+        "model.layers.24.attention.k_norm.weight": "model.safetensors",
+        "model.layers.24.attention.q_norm.weight": "model.safetensors",
+        "model.layers.24.attention.wo.biases": "model.safetensors",
+        "model.layers.24.attention.wo.scales": "model.safetensors",
+        "model.layers.24.attention.wo.weight": "model.safetensors",
+        "model.layers.24.attention.wqkv.biases": "model.safetensors",
+        "model.layers.24.attention.wqkv.scales": "model.safetensors",
+        "model.layers.24.attention.wqkv.weight": "model.safetensors",
+        "model.layers.24.attention_norm.weight": "model.safetensors",
+        "model.layers.24.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.24.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.24.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.24.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.24.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.24.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.24.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.24.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.24.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.24.ffn_norm.weight": "model.safetensors",
+        "model.layers.25.attention.k_norm.weight": "model.safetensors",
+        "model.layers.25.attention.q_norm.weight": "model.safetensors",
+        "model.layers.25.attention.wo.biases": "model.safetensors",
+        "model.layers.25.attention.wo.scales": "model.safetensors",
+        "model.layers.25.attention.wo.weight": "model.safetensors",
+        "model.layers.25.attention.wqkv.biases": "model.safetensors",
+        "model.layers.25.attention.wqkv.scales": "model.safetensors",
+        "model.layers.25.attention.wqkv.weight": "model.safetensors",
+        "model.layers.25.attention_norm.weight": "model.safetensors",
+        "model.layers.25.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.25.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.25.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.25.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.25.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.25.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.25.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.25.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.25.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.25.ffn_norm.weight": "model.safetensors",
+        "model.layers.26.attention.k_norm.weight": "model.safetensors",
+        "model.layers.26.attention.q_norm.weight": "model.safetensors",
+        "model.layers.26.attention.wo.biases": "model.safetensors",
+        "model.layers.26.attention.wo.scales": "model.safetensors",
+        "model.layers.26.attention.wo.weight": "model.safetensors",
+        "model.layers.26.attention.wqkv.biases": "model.safetensors",
+        "model.layers.26.attention.wqkv.scales": "model.safetensors",
+        "model.layers.26.attention.wqkv.weight": "model.safetensors",
+        "model.layers.26.attention_norm.weight": "model.safetensors",
+        "model.layers.26.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.26.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.26.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.26.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.26.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.26.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.26.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.26.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.26.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.26.ffn_norm.weight": "model.safetensors",
+        "model.layers.27.attention.k_norm.weight": "model.safetensors",
+        "model.layers.27.attention.q_norm.weight": "model.safetensors",
+        "model.layers.27.attention.wo.biases": "model.safetensors",
+        "model.layers.27.attention.wo.scales": "model.safetensors",
+        "model.layers.27.attention.wo.weight": "model.safetensors",
+        "model.layers.27.attention.wqkv.biases": "model.safetensors",
+        "model.layers.27.attention.wqkv.scales": "model.safetensors",
+        "model.layers.27.attention.wqkv.weight": "model.safetensors",
+        "model.layers.27.attention_norm.weight": "model.safetensors",
+        "model.layers.27.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.27.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.27.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.27.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.27.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.27.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.27.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.27.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.27.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.27.ffn_norm.weight": "model.safetensors",
+        "model.layers.28.attention.k_norm.weight": "model.safetensors",
+        "model.layers.28.attention.q_norm.weight": "model.safetensors",
+        "model.layers.28.attention.wo.biases": "model.safetensors",
+        "model.layers.28.attention.wo.scales": "model.safetensors",
+        "model.layers.28.attention.wo.weight": "model.safetensors",
+        "model.layers.28.attention.wqkv.biases": "model.safetensors",
+        "model.layers.28.attention.wqkv.scales": "model.safetensors",
+        "model.layers.28.attention.wqkv.weight": "model.safetensors",
+        "model.layers.28.attention_norm.weight": "model.safetensors",
+        "model.layers.28.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.28.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.28.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.28.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.28.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.28.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.28.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.28.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.28.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.28.ffn_norm.weight": "model.safetensors",
+        "model.layers.29.attention.k_norm.weight": "model.safetensors",
+        "model.layers.29.attention.q_norm.weight": "model.safetensors",
+        "model.layers.29.attention.wo.biases": "model.safetensors",
+        "model.layers.29.attention.wo.scales": "model.safetensors",
+        "model.layers.29.attention.wo.weight": "model.safetensors",
+        "model.layers.29.attention.wqkv.biases": "model.safetensors",
+        "model.layers.29.attention.wqkv.scales": "model.safetensors",
+        "model.layers.29.attention.wqkv.weight": "model.safetensors",
+        "model.layers.29.attention_norm.weight": "model.safetensors",
+        "model.layers.29.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.29.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.29.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.29.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.29.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.29.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.29.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.29.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.29.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.29.ffn_norm.weight": "model.safetensors",
+        "model.layers.3.attention.k_norm.weight": "model.safetensors",
+        "model.layers.3.attention.q_norm.weight": "model.safetensors",
+        "model.layers.3.attention.wo.biases": "model.safetensors",
+        "model.layers.3.attention.wo.scales": "model.safetensors",
+        "model.layers.3.attention.wo.weight": "model.safetensors",
+        "model.layers.3.attention.wqkv.biases": "model.safetensors",
+        "model.layers.3.attention.wqkv.scales": "model.safetensors",
+        "model.layers.3.attention.wqkv.weight": "model.safetensors",
+        "model.layers.3.attention_norm.weight": "model.safetensors",
+        "model.layers.3.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.3.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.3.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.3.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.3.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.3.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.3.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.3.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.3.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.3.ffn_norm.weight": "model.safetensors",
+        "model.layers.30.attention.k_norm.weight": "model.safetensors",
+        "model.layers.30.attention.q_norm.weight": "model.safetensors",
+        "model.layers.30.attention.wo.biases": "model.safetensors",
+        "model.layers.30.attention.wo.scales": "model.safetensors",
+        "model.layers.30.attention.wo.weight": "model.safetensors",
+        "model.layers.30.attention.wqkv.biases": "model.safetensors",
+        "model.layers.30.attention.wqkv.scales": "model.safetensors",
+        "model.layers.30.attention.wqkv.weight": "model.safetensors",
+        "model.layers.30.attention_norm.weight": "model.safetensors",
+        "model.layers.30.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.30.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.30.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.30.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.30.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.30.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.30.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.30.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.30.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.30.ffn_norm.weight": "model.safetensors",
+        "model.layers.31.attention.k_norm.weight": "model.safetensors",
+        "model.layers.31.attention.q_norm.weight": "model.safetensors",
+        "model.layers.31.attention.wo.biases": "model.safetensors",
+        "model.layers.31.attention.wo.scales": "model.safetensors",
+        "model.layers.31.attention.wo.weight": "model.safetensors",
+        "model.layers.31.attention.wqkv.biases": "model.safetensors",
+        "model.layers.31.attention.wqkv.scales": "model.safetensors",
+        "model.layers.31.attention.wqkv.weight": "model.safetensors",
+        "model.layers.31.attention_norm.weight": "model.safetensors",
+        "model.layers.31.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.31.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.31.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.31.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.31.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.31.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.31.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.31.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.31.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.31.ffn_norm.weight": "model.safetensors",
+        "model.layers.32.attention.k_norm.weight": "model.safetensors",
+        "model.layers.32.attention.q_norm.weight": "model.safetensors",
+        "model.layers.32.attention.wo.biases": "model.safetensors",
+        "model.layers.32.attention.wo.scales": "model.safetensors",
+        "model.layers.32.attention.wo.weight": "model.safetensors",
+        "model.layers.32.attention.wqkv.biases": "model.safetensors",
+        "model.layers.32.attention.wqkv.scales": "model.safetensors",
+        "model.layers.32.attention.wqkv.weight": "model.safetensors",
+        "model.layers.32.attention_norm.weight": "model.safetensors",
+        "model.layers.32.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.32.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.32.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.32.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.32.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.32.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.32.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.32.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.32.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.32.ffn_norm.weight": "model.safetensors",
+        "model.layers.33.attention.k_norm.weight": "model.safetensors",
+        "model.layers.33.attention.q_norm.weight": "model.safetensors",
+        "model.layers.33.attention.wo.biases": "model.safetensors",
+        "model.layers.33.attention.wo.scales": "model.safetensors",
+        "model.layers.33.attention.wo.weight": "model.safetensors",
+        "model.layers.33.attention.wqkv.biases": "model.safetensors",
+        "model.layers.33.attention.wqkv.scales": "model.safetensors",
+        "model.layers.33.attention.wqkv.weight": "model.safetensors",
+        "model.layers.33.attention_norm.weight": "model.safetensors",
+        "model.layers.33.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.33.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.33.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.33.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.33.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.33.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.33.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.33.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.33.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.33.ffn_norm.weight": "model.safetensors",
+        "model.layers.34.attention.k_norm.weight": "model.safetensors",
+        "model.layers.34.attention.q_norm.weight": "model.safetensors",
+        "model.layers.34.attention.wo.biases": "model.safetensors",
+        "model.layers.34.attention.wo.scales": "model.safetensors",
+        "model.layers.34.attention.wo.weight": "model.safetensors",
+        "model.layers.34.attention.wqkv.biases": "model.safetensors",
+        "model.layers.34.attention.wqkv.scales": "model.safetensors",
+        "model.layers.34.attention.wqkv.weight": "model.safetensors",
+        "model.layers.34.attention_norm.weight": "model.safetensors",
+        "model.layers.34.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.34.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.34.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.34.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.34.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.34.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.34.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.34.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.34.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.34.ffn_norm.weight": "model.safetensors",
+        "model.layers.35.attention.k_norm.weight": "model.safetensors",
+        "model.layers.35.attention.q_norm.weight": "model.safetensors",
+        "model.layers.35.attention.wo.biases": "model.safetensors",
+        "model.layers.35.attention.wo.scales": "model.safetensors",
+        "model.layers.35.attention.wo.weight": "model.safetensors",
+        "model.layers.35.attention.wqkv.biases": "model.safetensors",
+        "model.layers.35.attention.wqkv.scales": "model.safetensors",
+        "model.layers.35.attention.wqkv.weight": "model.safetensors",
+        "model.layers.35.attention_norm.weight": "model.safetensors",
+        "model.layers.35.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.35.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.35.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.35.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.35.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.35.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.35.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.35.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.35.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.35.ffn_norm.weight": "model.safetensors",
+        "model.layers.4.attention.k_norm.weight": "model.safetensors",
+        "model.layers.4.attention.q_norm.weight": "model.safetensors",
+        "model.layers.4.attention.wo.biases": "model.safetensors",
+        "model.layers.4.attention.wo.scales": "model.safetensors",
+        "model.layers.4.attention.wo.weight": "model.safetensors",
+        "model.layers.4.attention.wqkv.biases": "model.safetensors",
+        "model.layers.4.attention.wqkv.scales": "model.safetensors",
+        "model.layers.4.attention.wqkv.weight": "model.safetensors",
+        "model.layers.4.attention_norm.weight": "model.safetensors",
+        "model.layers.4.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.4.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.4.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.4.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.4.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.4.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.4.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.4.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.4.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.4.ffn_norm.weight": "model.safetensors",
+        "model.layers.5.attention.k_norm.weight": "model.safetensors",
+        "model.layers.5.attention.q_norm.weight": "model.safetensors",
+        "model.layers.5.attention.wo.biases": "model.safetensors",
+        "model.layers.5.attention.wo.scales": "model.safetensors",
+        "model.layers.5.attention.wo.weight": "model.safetensors",
+        "model.layers.5.attention.wqkv.biases": "model.safetensors",
+        "model.layers.5.attention.wqkv.scales": "model.safetensors",
+        "model.layers.5.attention.wqkv.weight": "model.safetensors",
+        "model.layers.5.attention_norm.weight": "model.safetensors",
+        "model.layers.5.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.5.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.5.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.5.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.5.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.5.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.5.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.5.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.5.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.5.ffn_norm.weight": "model.safetensors",
+        "model.layers.6.attention.k_norm.weight": "model.safetensors",
+        "model.layers.6.attention.q_norm.weight": "model.safetensors",
+        "model.layers.6.attention.wo.biases": "model.safetensors",
+        "model.layers.6.attention.wo.scales": "model.safetensors",
+        "model.layers.6.attention.wo.weight": "model.safetensors",
+        "model.layers.6.attention.wqkv.biases": "model.safetensors",
+        "model.layers.6.attention.wqkv.scales": "model.safetensors",
+        "model.layers.6.attention.wqkv.weight": "model.safetensors",
+        "model.layers.6.attention_norm.weight": "model.safetensors",
+        "model.layers.6.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.6.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.6.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.6.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.6.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.6.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.6.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.6.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.6.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.6.ffn_norm.weight": "model.safetensors",
+        "model.layers.7.attention.k_norm.weight": "model.safetensors",
+        "model.layers.7.attention.q_norm.weight": "model.safetensors",
+        "model.layers.7.attention.wo.biases": "model.safetensors",
+        "model.layers.7.attention.wo.scales": "model.safetensors",
+        "model.layers.7.attention.wo.weight": "model.safetensors",
+        "model.layers.7.attention.wqkv.biases": "model.safetensors",
+        "model.layers.7.attention.wqkv.scales": "model.safetensors",
+        "model.layers.7.attention.wqkv.weight": "model.safetensors",
+        "model.layers.7.attention_norm.weight": "model.safetensors",
+        "model.layers.7.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.7.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.7.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.7.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.7.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.7.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.7.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.7.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.7.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.7.ffn_norm.weight": "model.safetensors",
+        "model.layers.8.attention.k_norm.weight": "model.safetensors",
+        "model.layers.8.attention.q_norm.weight": "model.safetensors",
+        "model.layers.8.attention.wo.biases": "model.safetensors",
+        "model.layers.8.attention.wo.scales": "model.safetensors",
+        "model.layers.8.attention.wo.weight": "model.safetensors",
+        "model.layers.8.attention.wqkv.biases": "model.safetensors",
+        "model.layers.8.attention.wqkv.scales": "model.safetensors",
+        "model.layers.8.attention.wqkv.weight": "model.safetensors",
+        "model.layers.8.attention_norm.weight": "model.safetensors",
+        "model.layers.8.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.8.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.8.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.8.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.8.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.8.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.8.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.8.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.8.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.8.ffn_norm.weight": "model.safetensors",
+        "model.layers.9.attention.k_norm.weight": "model.safetensors",
+        "model.layers.9.attention.q_norm.weight": "model.safetensors",
+        "model.layers.9.attention.wo.biases": "model.safetensors",
+        "model.layers.9.attention.wo.scales": "model.safetensors",
+        "model.layers.9.attention.wo.weight": "model.safetensors",
+        "model.layers.9.attention.wqkv.biases": "model.safetensors",
+        "model.layers.9.attention.wqkv.scales": "model.safetensors",
+        "model.layers.9.attention.wqkv.weight": "model.safetensors",
+        "model.layers.9.attention_norm.weight": "model.safetensors",
+        "model.layers.9.feed_forward.w1.biases": "model.safetensors",
+        "model.layers.9.feed_forward.w1.scales": "model.safetensors",
+        "model.layers.9.feed_forward.w1.weight": "model.safetensors",
+        "model.layers.9.feed_forward.w2.biases": "model.safetensors",
+        "model.layers.9.feed_forward.w2.scales": "model.safetensors",
+        "model.layers.9.feed_forward.w2.weight": "model.safetensors",
+        "model.layers.9.feed_forward.w3.biases": "model.safetensors",
+        "model.layers.9.feed_forward.w3.scales": "model.safetensors",
+        "model.layers.9.feed_forward.w3.weight": "model.safetensors",
+        "model.layers.9.ffn_norm.weight": "model.safetensors",
+        "model.norm.weight": "model.safetensors"
+    }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f24e08099d45a8adf3f52f5f0b03276e433bb9d689bb15fcbcc48ce58744588b
+size 12217872