Duplicate from emissary-ai/Python-Tab-Completion-CodeLlama-70b

Co-authored-by: Krish Desai <krish-emissary@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,87 @@
+
+---
+language:
+- code
+pipeline_tag: text-generation
+tags:
+- llama-2
+license: llama2
+---
+# **Code Llama**
+Code Llama is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. This is the repository for the 70B Python specialist version in the Hugging Face Transformers format. This model is designed for general code synthesis and understanding. Links to other models can be found in the index at the bottom.
+
+> [!NOTE]
+> This is a non-official Code Llama repo. You can find the official Meta repository in the [Meta Llama organization](https://huggingface.co/meta-llama/CodeLlama-70b-Python-hf).
+
+|     | Base Model                                                                    | Python                                                                                      | Instruct                                                                                        |
+| --- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
+| 7B  | [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) | [codellama/CodeLlama-7b-Python-hf](https://huggingface.co/codellama/CodeLlama-7b-Python-hf) | [codellama/CodeLlama-7b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf) |
+| 13B  | [codellama/CodeLlama-13b-hf](https://huggingface.co/codellama/CodeLlama-13b-hf) | [codellama/CodeLlama-13b-Python-hf](https://huggingface.co/codellama/CodeLlama-13b-Python-hf) | [codellama/CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf) |
+| 34B  | [codellama/CodeLlama-34b-hf](https://huggingface.co/codellama/CodeLlama-34b-hf) | [codellama/CodeLlama-34b-Python-hf](https://huggingface.co/codellama/CodeLlama-34b-Python-hf) | [codellama/CodeLlama-34b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf) |
+| 70B  | [codellama/CodeLlama-70b-hf](https://huggingface.co/codellama/CodeLlama-70b-hf) | [codellama/CodeLlama-70b-Python-hf](https://huggingface.co/codellama/CodeLlama-70b-Python-hf) | [codellama/CodeLlama-70b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-70b-Instruct-hf) |
+
+## Model Use
+
+To use this model, please make sure to install `transformers`. 
+
+```bash
+pip install transformers accelerate
+```
+
+Model capabilities:
+
+- [x] Code completion.
+- [ ] Infilling.
+- [ ] Instructions / chat.
+- [x] Python specialist.
+
+## Model Details
+*Note: Use of this model is governed by the Meta license. Meta developed and publicly released the Code Llama family of large language models (LLMs).
+
+**Model Developers** Meta
+
+**Variations** Code Llama comes in four model sizes, and three variants:
+
+* Code Llama: base models designed for general code synthesis and understanding
+* Code Llama - Python: designed specifically for Python
+* Code Llama - Instruct: for instruction following and safer deployment
+
+All variants are available in sizes of 7B, 13B, 34B, and 70B parameters.
+
+**This repository contains the Python version of the 70B parameters model.**
+
+**Input** Models input text only.
+
+**Output** Models generate text only.
+
+**Model Architecture** Code Llama is an auto-regressive language model that uses an optimized transformer architecture.
+**Model Architecture** Code Llama is an auto-regressive language model that uses an optimized transformer architecture. It was fine-tuned with up to 16k tokens. This variant **does not** support long context of up to 100k tokens.
+
+**Model Dates** Code Llama and its variants have been trained between January 2023 and January 2024.
+
+**Status** This is a static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as we improve model safety with community feedback.
+
+**License** A custom commercial license is available at: [https://ai.meta.com/resources/models-and-libraries/llama-downloads/](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)
+
+**Research Paper** More information can be found in the paper "[Code Llama: Open Foundation Models for Code](https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/)" or its [arXiv page](https://arxiv.org/abs/2308.12950).
+
+## Intended Use
+**Intended Use Cases** Code Llama and its variants are intended for commercial and research use in English and relevant programming languages. The base model Code Llama can be adapted for a variety of code synthesis and understanding tasks, Code Llama - Python is designed specifically to handle the Python programming language, and Code Llama - Instruct is intended to be safer to use for code assistant and generation applications.
+
+**Out-of-Scope Uses** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English. Use in any other way that is prohibited by the Acceptable Use Policy and Licensing Agreement for Code Llama and its variants.
+
+## Hardware and Software
+**Training Factors** We used custom training libraries. The training and fine-tuning of the released models have been performed Meta’s Research Super Cluster.
+
+**Carbon Footprint** In aggregate, training all 12 Code Llama models required 1400K GPU hours of computation on hardware of type A100-80GB (TDP of 350-400W). Estimated total emissions were 228.55 tCO2eq, 100% of which were offset by Meta’s sustainability program.
+
+## Evaluation Results
+
+See evaluations for the main models and detailed ablations in Section 3 and safety evaluations in Section 4 of the research paper.
+
+
+## Ethical Considerations and Limitations
+
+Code Llama and its variants are a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Code Llama’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. Therefore, before deploying any applications of Code Llama, developers should perform safety testing and tuning tailored to their specific applications of the model.
+
+Please see the Responsible Use Guide available available at [https://ai.meta.com/llama/responsible-use-guide](https://ai.meta.com/llama/responsible-use-guide).

config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.37.1",
+  "use_cache": true,
+  "vocab_size": 32016
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.37.1"
+}

model-00001-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0dc39667e9e9bb99b37e164514e2517c859a21b69083d1e391c95454538b8fc0
+size 4718922480

model-00002-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93cb9a3be35a63540022de4c9e0d19f48310ea491ebd11f077575c0907b523d7
+size 4664167376

model-00003-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:804d76bf0780390092e0ad15338b40b82221f5a89df019e58c7ef71ddf953433
+size 4999711704

model-00004-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15e3751d89cdda1ed46ca2b27e66ef527fa2937b69ffe8f52cf0a30a0ad13038
+size 4966157048

model-00005-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d857c396d675c945cfc4c6149269f475dc60f1ff38810f446be373e5ad0ca402
+size 4664134408

model-00006-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f325631196d211f66f4b614d7ef7733869da094044732b48399da5f14dc6992
+size 4664167408

model-00007-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49e617ff57141a9cd574fb68738d8cc7700d8732f3f793b6e6b5ffd99967f856
+size 4664167408

model-00008-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61089d0168ff2c89407d8d8aec42572ed542ad1c0d8c16a1c0199f56bcb8f886
+size 4999711728

model-00009-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10339c063a9708cadde61d3a690d1a2b65cbdb66a3af02f762fa6220379e9494
+size 4966157056

model-00010-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44cf1bc764a3183a1a30478399dd14810da918dc861f1f3f8ca5722e3051aa5c
+size 4664134408

model-00011-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:850147b678c3950665c18042ed185022d8c18b9e7706a08f5b56a3e165ea1d08
+size 4664167408

model-00012-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:054899a12a20c0014e304b1f69260676d4ee7b6437789d3a06eabd48588ffe0b
+size 4664167408

model-00013-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebf60aa2ee9a142b54d164c207340aa6fc2633e9986d54fe820b3fff7abfc75a
+size 4999711728

model-00014-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99d9ec085297c0e9771261376e128487b7e7d8cb8ff48ca68cd1004d2f99ceba
+size 4966157056

model-00015-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:665b12bb855f6a1afce80e1255fbceebf683923498d048ebaaced0dade3882e6
+size 4664134408

model-00016-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07ec206dd2ad1cc5c5c939eff06ea6858c30a51b6efda3390da5ef49ddf981e4
+size 4664167408

model-00017-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbdbc85b01918662674b27345b64439a494781f1b892f245db57f9162cb1f887
+size 4664167408

model-00018-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5309faee1ab036dfbe8103f4aa8c2dba77988e9a2c5c92fd5f00580f650f486
+size 4999711728

model-00019-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:140288ba636559cef56179e649253997f8fa72437717b21bc060262bc3416468
+size 4966157056

model-00020-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7466b2e9e116445857d71019cef746dbc63176b5568ad600da48a08294cb63e
+size 4664134408

model-00021-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d28bcf68247f93c5b70626b917aa0349103feaedd267714b6e4e9bc5ac65b689
+size 4664167408

model-00022-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d9419b0dbd0fd6a8784180d8aca472fc922cecefb03083b9f8371a86c708097
+size 4664167408

model-00023-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b0b88e28c69c3f2245d5b68f11832e2161f578881ed7f6a8dfe5f01d53051d3
+size 4999711728

model-00024-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed64d23c1c38899c2ff00e1d3bf1f73239ca06b7d691878668dbe31cbee115a8
+size 4966157056

model-00025-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b93038be5716e03c6304a7574dd0cb90f65ebb54fb519669c5046ae1249060b5
+size 4664134408

model-00026-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf37ac601e61c9d476cd2edda9b2f95d7a3c89e9d90a6226f455758b891c8dd1
+size 4664167408

model-00027-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f561fea5218a2cb49f210748bf766dfe63603912e8950f0a7f5d4ddaa5d0512a
+size 4664167408

model-00028-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1001859efc68cdefe27f45f9910af9ff8ea7c864124270e2fcfc6038c22b531b
+size 4999711728

model-00029-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3129bb872e06ec49f86d54fafeabb402787835540aee51300bb9586c79df2c07
+size 3779413928

model.safetensors.index.json

@@ -0,0 +1,730 @@
+{
+    "metadata": {
+        "total_size": 137953820672
+    },
+    "weight_map": {
+        "lm_head.weight": "model-00029-of-00029.safetensors",
+        "model.embed_tokens.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.input_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.down_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.mlp.up_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.input_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.down_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.mlp.up_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.10.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.10.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.11.input_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.11.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.input_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.12.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.13.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.13.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+        "model.layers.14.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.14.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.input_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.15.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.16.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.16.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+        "model.layers.17.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.17.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.input_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.18.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.19.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.19.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+        "model.layers.2.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+        "model.layers.20.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.20.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.input_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.21.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.22.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.22.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+        "model.layers.23.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.23.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.input_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.24.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+        "model.layers.25.input_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.25.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.input_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.26.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.27.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.27.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+        "model.layers.28.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.28.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.input_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.29.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.3.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.30.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.30.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.30.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+        "model.layers.31.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.31.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.input_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.32.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.33.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.33.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+        "model.layers.34.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.34.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.input_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.35.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.36.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.36.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+        "model.layers.37.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.37.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.input_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.38.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+        "model.layers.39.input_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.39.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.4.input_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.40.input_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.40.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.41.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.41.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+        "model.layers.42.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.42.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.input_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.43.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.44.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.44.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+        "model.layers.45.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.45.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.input_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.46.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.47.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.47.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+        "model.layers.48.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.48.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.input_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.49.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.5.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+        "model.layers.50.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.50.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.50.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.50.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+        "model.layers.51.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.51.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.input_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.52.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+        "model.layers.53.input_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.53.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.input_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.54.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.55.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.55.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+        "model.layers.56.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.56.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.input_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.57.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.58.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.58.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+        "model.layers.59.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.59.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.6.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.6.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.60.input_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.60.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.61.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.61.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+        "model.layers.62.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.62.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.input_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.63.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.64.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.64.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+        "model.layers.65.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.65.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.input_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.66.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+        "model.layers.67.input_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.67.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.input_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.68.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.69.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.69.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+        "model.layers.7.input_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.7.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.70.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.70.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.input_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.71.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.72.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.72.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+        "model.layers.73.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.73.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.input_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.74.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.75.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.75.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+        "model.layers.76.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.76.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.input_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.77.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.input_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.down_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.gate_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.mlp.up_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.post_attention_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.self_attn.o_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.78.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.78.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+        "model.layers.79.input_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.down_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.gate_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.mlp.up_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.post_attention_layernorm.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.k_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.o_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.q_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.79.self_attn.v_proj.weight": "model-00029-of-00029.safetensors",
+        "model.layers.8.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.8.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+        "model.layers.9.input_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+        "model.layers.9.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+        "model.norm.weight": "model-00029-of-00029.safetensors"
+    }
+}

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3f28cf15889c0bc501b6613bd6affeb7c8fbb38fd5fcd3caa51dd859df4e251
+size 141675740968

pytorch_model-00001-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4b2b2319420e9b426c45d40c188d438758c0dbe29e878107966a81209667df5
+size 4718928338

pytorch_model-00002-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dee546284bc9c43eae1a45a83f08f3c832ae2ee1ade02e2ad0833cced9c76bc7
+size 4664173598

pytorch_model-00003-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01285a3719d7bacd443e6ea35158fc4a877a73e65005624f6b513b3fd162139c
+size 4999717966

pytorch_model-00004-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:584d439f395de303d39931b1e88ff5f4871d06f8a80c8fe29866727a1f0a4390
+size 4966162946

pytorch_model-00005-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:169fe45301b4e0e33d272c99faa9d666f7defc7c09a00d904932f2021874cce0
+size 4664140254

pytorch_model-00006-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe97b617d794ed1300ebdbe0bc9a9b2b0d4f7056ed130b757b394b8dd3002e0d
+size 4664173662

pytorch_model-00007-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:018be562bf5e81e6b6d7894a12c1d0319a24599cfe860c78127730740425cc46
+size 4664173662

pytorch_model-00008-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d2820934d03c59678c2bc5d084f7ac278d49ba9803276d021d221d809c1caef
+size 4999718030

pytorch_model-00009-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cf080d1d082f8a5c5ba9ecc6b132e9ebac2d6b666f611a7b501375cf5b852b9
+size 4966162946

pytorch_model-00010-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b932c647486d35f9504e536cc4adddf4a0024349a2761bc5513e7ecfa7b75108
+size 4664140254

pytorch_model-00011-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7be53afe83fac9fdb36d7374192d65eaadcea237a829a3a53a1e74e7a07025c9
+size 4664173662

pytorch_model-00012-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe76a0b9432fe65344dae2b81b7817a6b57b10162f6506b2e5182c4cc706f442
+size 4664173662

pytorch_model-00013-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc1b1db8614091dc9810138422f7c188707e4f0b50f0d790a6073c4a7b80b4a3
+size 4999718030

pytorch_model-00014-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c032cd8c883adb0226a8d4248090f90ea9b4ed56f02047636c52608048c87086
+size 4966162946

pytorch_model-00015-of-00029.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e92e860739c83d248e238f8d9638734cd7f80e36191c3fac38e32eb4492241ed
+size 4664140254