Add files using upload-large-folder tool

README.md

@@ -0,0 +1,114 @@
+---
+license: other
+library_name: mlx
+pipeline_tag: text-to-audio
+base_model:
+- tencent/SongGeneration
+tags:
+- mlx
+- apple-silicon
+- music-generation
+- song-generation
+- audiolm
+---
+
+Part of the SongGeneration MLX conversion set.
+
+# SongGeneration-v2-medium-8bit
+
+Apple MLX weights for the autoregressive `audiolm` token generator from Tencent SongGeneration v2-medium.
+
+This is not a full-stack pure MLX audio pipeline yet: token generation runs with MLX, while FLAC decoding currently uses the official PyTorch Flow1dVAE / separate-tokenizer bridge in [`ailuntx/SongGeneration-MLX`](https://github.com/ailuntx/SongGeneration-MLX).
+
+## TL;DR
+
+| | |
+|---|---|
+| **Variant** | `v2-medium` |
+| **Precision** | `8bit` |
+| **Converted component** | SongGeneration `audiolm` token generator |
+| **Runtime** | [`ailuntx/SongGeneration-MLX`](https://github.com/ailuntx/SongGeneration-MLX) |
+| **Official model** | [`tencent/SongGeneration`](https://huggingface.co/tencent/SongGeneration) |
+| **Official code** | [`tencent-ailab/songgeneration`](https://github.com/tencent-ailab/songgeneration) |
+
+## Quick Start
+
+```bash
+hf download mlx-community/SongGeneration-v2-medium-8bit --local-dir ./models/SongGeneration-v2-medium-8bit
+
+git clone https://github.com/ailuntx/SongGeneration-MLX.git
+cd SongGeneration-MLX
+python -m venv .venv
+.venv/bin/pip install -e .
+
+PYTHONPATH=. .venv/bin/python -m songgeneration_mlx.cli \
+  --model ./models/SongGeneration-v2-medium-8bit \
+  --lyrics '[verse] hello from mlx [chorus] sing it again' \
+  --description 'Pop, female vocal, bright production, [Musicality-medium].' \
+  --duration 2 \
+  --top-k 50 \
+  --temperature 0.9 \
+  --output tokens_2s.npz
+```
+
+To decode tokens to FLAC, use the official decoder bridge documented in the GitHub README. The bridge requires the official SongGeneration runtime assets.
+
+## Variants
+
+| Variant | Disk | Notes |
+|---|---:|---|
+| `SongGeneration-v2-medium-fp32` | 10G | high-precision medium baseline |
+| `SongGeneration-v2-medium-bfloat16` | 5.2G | recommended medium quality baseline |
+| `SongGeneration-v2-medium-8bit` | 2.8G | smaller medium checkpoint |
+| `SongGeneration-v2-medium-4bit` | 1.5G | smallest medium checkpoint |
+| `SongGeneration-v2-large-fp32` | 19G | high-precision large baseline |
+| `SongGeneration-v2-large-bfloat16` | 9.5G | large quality baseline |
+| `SongGeneration-v2-large-8bit` | 5.0G | smaller large checkpoint |
+| `SongGeneration-v2-large-4bit` | 2.7G | smallest large checkpoint |
+
+## Layout
+
+```text
+SongGeneration-v2-medium-8bit/
+|-- model-00001-of-000xx.safetensors
+|-- model.safetensors.index.json
+|-- config.json
+|-- mlx_manifest.json
+|-- config.official.yaml
+|-- vocab.yaml
+`-- qwen2_tokenizer/
+```
+
+## Validation
+
+Local Apple Silicon validation was run on the medium bfloat16 path:
+
+| Test | Result |
+|---|---|
+| 12s MLX token generation | 550 pattern steps, about 1 minute wall time |
+| 12s official decoder bridge | 73.27s wall time |
+| 12s FLAC | 48kHz stereo, 12.000s, RMS about `0.163` |
+
+The official recent-token repetition penalty is implemented in the MLX runtime. Without it, 12s generations collapse into repeated tokens and decode close to silence.
+
+## License
+
+License follows the upstream SongGeneration release. Check the official model card and repository for the authoritative model license.
+
+## Citation
+
+```bibtex
+@misc{songgeneration-mlx,
+  title  = {SongGeneration-MLX: Apple MLX port of SongGeneration},
+  author = {ailuntx},
+  year   = {2026},
+  url    = {https://github.com/ailuntx/SongGeneration-MLX},
+}
+
+@article{lei2025levo,
+  title   = {LeVo: High-Quality Song Generation with Multi-Preference Alignment},
+  author  = {Lei, Shun and Xu, Yaoxun and Lin, Zhiwei and Zhang, Huaicheng and Tan, Wei and Chen, Hangting and Yu, Jianwei and Zhang, Yixuan and Yang, Chenyu and Zhu, Haina and Wang, Shuai and Wu, Zhiyong and Yu, Dong},
+  journal = {arXiv preprint arXiv:2506.07520},
+  year    = {2025},
+}
+```

config.json

@@ -0,0 +1,89 @@
+{
+  "format": "songgeneration-mlx-audiolm",
+  "source": "tencent/SongGeneration",
+  "official_code": "https://github.com/tencent-ailab/songgeneration",
+  "variant": "v2-medium",
+  "precision": "8bit",
+  "runtime": {
+    "hidden_size": 1536,
+    "intermediate_size": 8960,
+    "num_heads": 12,
+    "num_layers": 28,
+    "num_layers_sub": 12,
+    "code_depth": 3,
+    "code_size": 16384,
+    "prompt_len": 10,
+    "frame_rate": 25,
+    "max_position_embeddings": 10000,
+    "max_position_embeddings_sub": 10000,
+    "rope_theta": 500000.0,
+    "rope_theta_sub": 500000.0,
+    "cfg_coef": 1.5
+  },
+  "components": {
+    "audiolm": "converted to MLX safetensors",
+    "qwen_tokenizer": "copied for lyric/style tokenization",
+    "audio_decoder": "use official PyTorch Flow1dVAE/separate tokenizer bridge for now"
+  },
+  "skipped_keys": [
+    "audiolm.out_norm.weight",
+    "audiolm.out_norm.bias",
+    "audiolm.transformer2.lm_head.weight"
+  ],
+  "quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine"
+  },
+  "weight_files": [
+    "model-00001-of-00049.safetensors",
+    "model-00002-of-00049.safetensors",
+    "model-00003-of-00049.safetensors",
+    "model-00004-of-00049.safetensors",
+    "model-00005-of-00049.safetensors",
+    "model-00006-of-00049.safetensors",
+    "model-00007-of-00049.safetensors",
+    "model-00008-of-00049.safetensors",
+    "model-00009-of-00049.safetensors",
+    "model-00010-of-00049.safetensors",
+    "model-00011-of-00049.safetensors",
+    "model-00012-of-00049.safetensors",
+    "model-00013-of-00049.safetensors",
+    "model-00014-of-00049.safetensors",
+    "model-00015-of-00049.safetensors",
+    "model-00016-of-00049.safetensors",
+    "model-00017-of-00049.safetensors",
+    "model-00018-of-00049.safetensors",
+    "model-00019-of-00049.safetensors",
+    "model-00020-of-00049.safetensors",
+    "model-00021-of-00049.safetensors",
+    "model-00022-of-00049.safetensors",
+    "model-00023-of-00049.safetensors",
+    "model-00024-of-00049.safetensors",
+    "model-00025-of-00049.safetensors",
+    "model-00026-of-00049.safetensors",
+    "model-00027-of-00049.safetensors",
+    "model-00028-of-00049.safetensors",
+    "model-00029-of-00049.safetensors",
+    "model-00030-of-00049.safetensors",
+    "model-00031-of-00049.safetensors",
+    "model-00032-of-00049.safetensors",
+    "model-00033-of-00049.safetensors",
+    "model-00034-of-00049.safetensors",
+    "model-00035-of-00049.safetensors",
+    "model-00036-of-00049.safetensors",
+    "model-00037-of-00049.safetensors",
+    "model-00038-of-00049.safetensors",
+    "model-00039-of-00049.safetensors",
+    "model-00040-of-00049.safetensors",
+    "model-00041-of-00049.safetensors",
+    "model-00042-of-00049.safetensors",
+    "model-00043-of-00049.safetensors",
+    "model-00044-of-00049.safetensors",
+    "model-00045-of-00049.safetensors",
+    "model-00046-of-00049.safetensors",
+    "model-00047-of-00049.safetensors",
+    "model-00048-of-00049.safetensors",
+    "model-00049-of-00049.safetensors"
+  ]
+}

config.official.yaml

@@ -0,0 +1,141 @@
+# ================ Train Config ================ #
+lyric_processor:
+max_dur: 270
+min_dur: 30
+prompt_len: 10
+pad_to_max: true
+
+
+# ================ Audio tokenzier ================ #
+audio_tokenizer_checkpoint: Flow1dVAE1rvq_./ckpt/model_1rvq/model_2_fixed.safetensors
+audio_tokenizer_frame_rate: 25
+audio_tokenizer_code_depth: 1
+sample_rate: 48000
+
+audio_tokenizer_checkpoint_sep: Flow1dVAESeparate_./ckpt/model_septoken/model_2.safetensors
+audio_tokenizer_frame_rate_sep: 25
+audio_tokenizer_code_depth_sep: 2
+sample_rate_sep: 48000
+
+# ================ VAE ================ #
+vae_config: ./ckpt/vae/stable_audio_1920_vae.json
+vae_model: ./ckpt/vae/autoencoder_music_1320k.ckpt
+
+# ================== LM =========================== #
+lm:
+  lm_type: Llama # [Llama]
+  dim: 1536
+  intermediate_size: 8960
+  num_heads: 12
+  num_layers: 28
+  num_layers_sub: 12
+  code_depth: 3
+  code_size: 16384
+  max_position_embeddings: 10000
+  max_position_embeddings_sub: 10000
+  rope_theta: 500000.0
+  rope_theta_sub: 500000.0
+  dropout: 0.0
+  use_flash_attn_2: true
+  activation: gelu
+  norm_first: true
+  bias_ff: false
+  bias_attn: false
+  causal: true
+  custom: false
+  memory_efficient: true
+  attention_as_float32: false
+  layer_scale: null
+  positional_embedding: sin
+  xpos: false
+  checkpointing: torch
+  weight_init: gaussian
+  depthwise_init: current
+  zero_bias_init: true
+  norm: layer_norm
+  cross_attention: false
+  qk_layer_norm: false
+  qk_layer_norm_cross: false
+  attention_dropout: null
+  kv_repeat: 1
+
+codebooks_pattern:
+  modeling: delay
+  delay:
+    delays: [ 0, 250, 250 ]
+    flatten_first: 0
+    empty_initial: 0
+
+# ================ Conditioners ===================== #
+classifier_free_guidance:
+  # drop all conditions simultaneously
+  training_dropout: 0.15
+  inference_coef: 1.5
+
+attribute_dropout:
+  # drop each condition separately
+  args:
+    active_on_eval: false
+  text:
+    description: 0.0
+    type_info: 0.2
+  audio:
+    prompt_audio: 0.5
+
+
+use_text_training: True
+fuser:
+  sum: []
+  prepend: [ description, prompt_audio, type_info ] # this order is the SAME with the input concatenation order
+
+conditioners:
+  prompt_audio:
+    model: qt_embedding
+    qt_embedding:
+      code_size: 16384
+      code_depth: 3
+      max_len: ${eval:${prompt_len}*${audio_tokenizer_frame_rate}+2} # 25*10+2+1
+  description:
+    model: QwTokenizer
+    QwTokenizer:
+      token_path: third_party/Qwen2-7B
+      max_len: 600
+      add_token_list: ${load_yaml:conf/vocab.yaml}
+  type_info:
+    model: QwTextTokenizer
+    QwTextTokenizer:
+      token_path: third_party/Qwen2-7B
+      max_len: 100
+
+offload:
+  audiolm:
+    offload_module: self
+    cpu_mem_gb: 0
+    pre_copy_step: 1
+    clean_cache_after_forward: false
+    dtype: torch.float16
+    offload_layer_dict:
+      transformer: 4
+      transformer2: 4
+    ignore_layer_list: []
+    clean_cache_wrapper:
+      module: self
+      method_name: _sample_next_token
+      diff_mem_gb_thre: 2
+    debug: false
+
+  wav_tokenizer_diffusion:
+    offload_module: self.model.model
+    pre_copy_step: 1
+    clean_cache_after_forward: false
+    cpu_mem_gb: -1
+    dtype: null
+    offload_layer_dict:
+      cfm_wrapper: 5
+      hubert: 4
+    ignore_layer_list: []
+    clean_cache_wrapper:
+      module: self.model.model.cfm_wrapper.estimator
+      method_name: forward
+      diff_mem_gb_thre: 1
+    debug: false

mlx_manifest.json

@@ -0,0 +1,89 @@
+{
+  "format": "songgeneration-mlx-audiolm",
+  "source": "tencent/SongGeneration",
+  "official_code": "https://github.com/tencent-ailab/songgeneration",
+  "variant": "v2-medium",
+  "precision": "8bit",
+  "runtime": {
+    "hidden_size": 1536,
+    "intermediate_size": 8960,
+    "num_heads": 12,
+    "num_layers": 28,
+    "num_layers_sub": 12,
+    "code_depth": 3,
+    "code_size": 16384,
+    "prompt_len": 10,
+    "frame_rate": 25,
+    "max_position_embeddings": 10000,
+    "max_position_embeddings_sub": 10000,
+    "rope_theta": 500000.0,
+    "rope_theta_sub": 500000.0,
+    "cfg_coef": 1.5
+  },
+  "components": {
+    "audiolm": "converted to MLX safetensors",
+    "qwen_tokenizer": "copied for lyric/style tokenization",
+    "audio_decoder": "use official PyTorch Flow1dVAE/separate tokenizer bridge for now"
+  },
+  "skipped_keys": [
+    "audiolm.out_norm.weight",
+    "audiolm.out_norm.bias",
+    "audiolm.transformer2.lm_head.weight"
+  ],
+  "quantization": {
+    "bits": 8,
+    "group_size": 64,
+    "mode": "affine"
+  },
+  "weight_files": [
+    "model-00001-of-00049.safetensors",
+    "model-00002-of-00049.safetensors",
+    "model-00003-of-00049.safetensors",
+    "model-00004-of-00049.safetensors",
+    "model-00005-of-00049.safetensors",
+    "model-00006-of-00049.safetensors",
+    "model-00007-of-00049.safetensors",
+    "model-00008-of-00049.safetensors",
+    "model-00009-of-00049.safetensors",
+    "model-00010-of-00049.safetensors",
+    "model-00011-of-00049.safetensors",
+    "model-00012-of-00049.safetensors",
+    "model-00013-of-00049.safetensors",
+    "model-00014-of-00049.safetensors",
+    "model-00015-of-00049.safetensors",
+    "model-00016-of-00049.safetensors",
+    "model-00017-of-00049.safetensors",
+    "model-00018-of-00049.safetensors",
+    "model-00019-of-00049.safetensors",
+    "model-00020-of-00049.safetensors",
+    "model-00021-of-00049.safetensors",
+    "model-00022-of-00049.safetensors",
+    "model-00023-of-00049.safetensors",
+    "model-00024-of-00049.safetensors",
+    "model-00025-of-00049.safetensors",
+    "model-00026-of-00049.safetensors",
+    "model-00027-of-00049.safetensors",
+    "model-00028-of-00049.safetensors",
+    "model-00029-of-00049.safetensors",
+    "model-00030-of-00049.safetensors",
+    "model-00031-of-00049.safetensors",
+    "model-00032-of-00049.safetensors",
+    "model-00033-of-00049.safetensors",
+    "model-00034-of-00049.safetensors",
+    "model-00035-of-00049.safetensors",
+    "model-00036-of-00049.safetensors",
+    "model-00037-of-00049.safetensors",
+    "model-00038-of-00049.safetensors",
+    "model-00039-of-00049.safetensors",
+    "model-00040-of-00049.safetensors",
+    "model-00041-of-00049.safetensors",
+    "model-00042-of-00049.safetensors",
+    "model-00043-of-00049.safetensors",
+    "model-00044-of-00049.safetensors",
+    "model-00045-of-00049.safetensors",
+    "model-00046-of-00049.safetensors",
+    "model-00047-of-00049.safetensors",
+    "model-00048-of-00049.safetensors",
+    "model-00049-of-00049.safetensors"
+  ]
+}

model-00001-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8ff9ce7d516fbd0d560ad601dedf1b7b0617f6944982249eaf0ca0fac7b03c1
+size 14559480

model-00004-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d241827ae9b066aa43c19c57ede56a3121738df6172886c47c3d451feea2a70
+size 53484360

model-00005-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a19e0a5d3678b71fe76112e27eb068f861daa4bf3f1b0636982409a38501ac49
+size 61008408

model-00007-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecee43f5971aba4d80ae18432ea3ef05bf7542b9d4f841dc61fad69b8c8add64
+size 66972408

model-00008-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e7485429669d86c10ca10cfb12cb35afd73d083ac892b81eb941cafafacb2a7
+size 54334176

model-00010-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bc35e90a859a81d12e8369acd9e21b334cbcc62c28c6731b64fe519882f6bd5
+size 53904000

model-00011-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54e10a7459dda82fc17c9e44fee1211b3972a24ce81905ccef9f0605e9d8cd08
+size 53904000

model-00012-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:095d6dd7cf7f4147c1662d04f38e9c37411154e9ba4a0b7b6fce68b0d76d87c0
+size 53904000

model-00013-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df48b585c33ae68349d24368694bb31532a759793d051cdab8756f2497643a41
+size 53904000

model-00015-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18b6c93d738d1cfe49dfde4cb2467463aae906bf93a605208d5e5dde8cc0fc75
+size 53904000

model-00016-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2bb5f65bf890937cbb911eabcd23ee868249f658b0967d0e811cfd35a19cf9c
+size 53904000

model-00017-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30b79d6bdce2b98e7f544768765e5bdda5948b7a06e3a16fae2c653bf4f45d6d
+size 53904000

model-00018-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0867da4d501ca9100173e49a21e50f599707e1b6ac46a8fd8c932f15b7ac7a6
+size 53904000

model-00019-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:394230dc4b3b25765157ef183f0a5a0fb5359101f23dc1f7cbf43b54fefec3d3
+size 53904000

model-00020-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2fae1c9c641e9ce09ff39a4649c48f302eea4b3793889acd36d6fb6cd060dc6
+size 53903984

model-00021-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7bacf716a1936d90105aa7a243072714cfceb0b072e055234694d3099467341
+size 53904000

model-00022-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81ee9bb9157de579cdca4a28a3f1fe0e3172d75dc31f0936c59bb5a998d41908
+size 53904000

model-00024-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e948d4194bde0d5dd5895563df13ea528ec2f427ad3e76f0a4c1423e8b077fba
+size 53904000

model-00025-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:470de7a0366cff07006c578bf2b49dcb3bcfb1f3cc5109582d65e407c2539b19
+size 53904000

model-00026-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6798a431f5bdfffccad69600d7397319796685091bfa3a95a0f5b351ee464e8d
+size 53904000

model-00027-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7b151217228b720a85ffe02d3ac98d6524a340177ed5ba02a32e304fda99e9e
+size 53904000

model-00028-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d89849b42538af84958eceab7de24e4bafe3708bf0d8b3d88b739870497354e
+size 53904000

model-00029-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09139083d846ec2e34527325e879016d94804c21db5080c9784243f2e02ea6dc
+size 53903976

model-00030-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05599f7000c165339733ab599d73e295451b2daacb80a2be4297e5e78a2f10c4
+size 53903976

model-00032-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2beb32d65f2eea4e9560719dc903e5181639aff9bf742eece757c4b218ee01f2
+size 53903976

model-00033-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1caa3b2b7e73de43122b71a1de777cad986f2a3bc13ed8cc58974dc553ff2065
+size 53903976

model-00035-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d6d5f95b4cadbec93628cb863ed3413a0fa87cb156e5752ea87024961062c2e
+size 66024720

model-00036-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84fe54219bfa401e95ab3f50cee35dfdcc79d64cfe1fd0f343f78f381b8ee062
+size 53904000

model-00037-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f05c79cc7a53668881f9bbf19ddb4c0563adbb1faae92acb3515658285366c7
+size 53904008

model-00038-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d763b1a1828a2c69a0edae313056f0bee2b2c881b348efa499940bdbf83d9a54
+size 53904024

model-00039-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3681ac9357e4aa62532009489197fe34ec2a6778a01f07da3812a595865fe05
+size 53904024

model-00040-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22afe0dfa79c352c03177d4cdc69e287bcaaa4164b4aaecb761fee6a3032db73
+size 53904000

model-00041-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7a4a3ecc74d0be53c298e3e061bf494f4dfc07b85dfbdac24c5a582afd930a9
+size 53904000

model-00043-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:caef4ebe9db66d25ddbd92ea0d38f126f15148f279be41f7f0df506fef3d524f
+size 53904000

model-00044-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d800067459f9427f9aee3a4d0c60b09ac06cc14f745c2c2ab81340beacccefc
+size 53904000

model-00045-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f306d60bbd8b90726bed9c0e218eee22920cb9aebc736ca6f8ef3646b6b75fbf
+size 53904000

model-00046-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c32c1a4280e990c970dc24b132494aa5257d1f07e7678635c435f769fa270b2
+size 53904000

model-00049-of-00049.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72beab245dc13bfa6720dda8a7203bc2fd98fcce3862b85d30615e1082fe72b
+size 232937584

model.safetensors.index.json

@@ -0,0 +1,984 @@
+{
+  "metadata": {
+    "total_size": 2979816384
+  },
+  "weight_map": {
+    "description_output_proj.biases": "model-00001-of-00049.safetensors",
+    "description_output_proj.scales": "model-00001-of-00049.safetensors",
+    "description_output_proj.weight": "model-00002-of-00049.safetensors",
+    "description_structure_emb.biases": "model-00003-of-00049.safetensors",
+    "description_structure_emb.scales": "model-00003-of-00049.safetensors",
+    "description_structure_emb.weight": "model-00003-of-00049.safetensors",
+    "emb0.biases": "model-00003-of-00049.safetensors",
+    "emb0.scales": "model-00003-of-00049.safetensors",
+    "emb0.weight": "model-00003-of-00049.safetensors",
+    "layer2_emb0.biases": "model-00003-of-00049.safetensors",
+    "layer2_emb0.scales": "model-00003-of-00049.safetensors",
+    "layer2_emb0.weight": "model-00003-of-00049.safetensors",
+    "layer2_emb1.biases": "model-00003-of-00049.safetensors",
+    "layer2_emb1.scales": "model-00003-of-00049.safetensors",
+    "layer2_emb1.weight": "model-00004-of-00049.safetensors",
+    "layer2_emb2.biases": "model-00004-of-00049.safetensors",
+    "layer2_emb2.scales": "model-00004-of-00049.safetensors",
+    "layer2_emb2.weight": "model-00004-of-00049.safetensors",
+    "linears0.biases": "model-00004-of-00049.safetensors",
+    "linears0.scales": "model-00004-of-00049.safetensors",
+    "linears0.weight": "model-00005-of-00049.safetensors",
+    "linears1.biases": "model-00005-of-00049.safetensors",
+    "linears1.scales": "model-00005-of-00049.safetensors",
+    "linears1.weight": "model-00005-of-00049.safetensors",
+    "mlp.linear0.bias": "model-00005-of-00049.safetensors",
+    "mlp.linear0.biases": "model-00005-of-00049.safetensors",
+    "mlp.linear0.scales": "model-00005-of-00049.safetensors",
+    "mlp.linear0.weight": "model-00005-of-00049.safetensors",
+    "mlp.linear2.bias": "model-00005-of-00049.safetensors",
+    "mlp.linear2.biases": "model-00005-of-00049.safetensors",
+    "mlp.linear2.scales": "model-00005-of-00049.safetensors",
+    "mlp.linear2.weight": "model-00005-of-00049.safetensors",
+    "prompt_audio_emb0.biases": "model-00005-of-00049.safetensors",
+    "prompt_audio_emb0.scales": "model-00005-of-00049.safetensors",
+    "prompt_audio_emb0.weight": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb1.biases": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb1.scales": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb1.weight": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb2.biases": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb2.scales": "model-00006-of-00049.safetensors",
+    "prompt_audio_emb2.weight": "model-00007-of-00049.safetensors",
+    "prompt_audio_eot": "model-00007-of-00049.safetensors",
+    "prompt_audio_layer2_eot": "model-00007-of-00049.safetensors",
+    "transformer.embed_tokens.biases": "model-00007-of-00049.safetensors",
+    "transformer.embed_tokens.scales": "model-00007-of-00049.safetensors",
+    "transformer.embed_tokens.weight": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.input_layernorm.weight": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.mlp.down_proj.biases": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.mlp.down_proj.scales": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.mlp.down_proj.weight": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.mlp.gate_proj.biases": "model-00007-of-00049.safetensors",
+    "transformer.layers.0.mlp.gate_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.mlp.gate_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.mlp.up_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.mlp.up_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.mlp.up_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.post_attention_layernorm.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.k_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.k_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.k_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.o_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.o_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.o_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.q_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.q_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.q_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.v_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.v_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.0.self_attn.v_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.input_layernorm.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.down_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.down_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.down_proj.weight": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.gate_proj.biases": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.gate_proj.scales": "model-00008-of-00049.safetensors",
+    "transformer.layers.1.mlp.gate_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.mlp.up_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.mlp.up_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.mlp.up_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.post_attention_layernorm.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.k_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.k_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.k_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.o_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.o_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.o_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.q_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.q_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.q_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.v_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.v_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.1.self_attn.v_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.input_layernorm.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.down_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.down_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.down_proj.weight": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.gate_proj.biases": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.gate_proj.scales": "model-00009-of-00049.safetensors",
+    "transformer.layers.10.mlp.gate_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.mlp.up_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.mlp.up_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.mlp.up_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.post_attention_layernorm.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.k_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.k_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.k_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.o_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.o_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.o_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.q_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.q_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.q_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.v_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.v_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.10.self_attn.v_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.input_layernorm.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.down_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.down_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.down_proj.weight": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.gate_proj.biases": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.gate_proj.scales": "model-00010-of-00049.safetensors",
+    "transformer.layers.11.mlp.gate_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.mlp.up_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.mlp.up_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.mlp.up_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.post_attention_layernorm.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.k_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.k_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.k_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.o_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.o_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.o_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.q_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.q_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.q_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.v_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.v_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.11.self_attn.v_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.input_layernorm.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.down_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.down_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.down_proj.weight": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.gate_proj.biases": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.gate_proj.scales": "model-00011-of-00049.safetensors",
+    "transformer.layers.12.mlp.gate_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.mlp.up_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.mlp.up_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.mlp.up_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.post_attention_layernorm.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.k_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.k_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.k_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.o_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.o_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.o_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.q_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.q_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.q_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.v_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.v_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.12.self_attn.v_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.input_layernorm.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.down_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.down_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.down_proj.weight": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.gate_proj.biases": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.gate_proj.scales": "model-00012-of-00049.safetensors",
+    "transformer.layers.13.mlp.gate_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.mlp.up_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.mlp.up_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.mlp.up_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.post_attention_layernorm.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.k_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.k_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.k_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.o_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.o_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.o_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.q_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.q_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.q_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.v_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.v_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.13.self_attn.v_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.input_layernorm.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.down_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.down_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.down_proj.weight": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.gate_proj.biases": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.gate_proj.scales": "model-00013-of-00049.safetensors",
+    "transformer.layers.14.mlp.gate_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.mlp.up_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.mlp.up_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.mlp.up_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.post_attention_layernorm.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.k_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.k_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.k_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.o_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.o_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.o_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.q_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.q_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.q_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.v_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.v_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.14.self_attn.v_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.input_layernorm.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.down_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.down_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.down_proj.weight": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.gate_proj.biases": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.gate_proj.scales": "model-00014-of-00049.safetensors",
+    "transformer.layers.15.mlp.gate_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.mlp.up_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.mlp.up_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.mlp.up_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.post_attention_layernorm.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.k_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.k_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.k_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.o_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.o_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.o_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.q_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.q_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.q_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.v_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.v_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.15.self_attn.v_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.input_layernorm.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.down_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.down_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.down_proj.weight": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.gate_proj.biases": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.gate_proj.scales": "model-00015-of-00049.safetensors",
+    "transformer.layers.16.mlp.gate_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.mlp.up_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.mlp.up_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.mlp.up_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.post_attention_layernorm.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.k_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.k_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.k_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.o_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.o_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.o_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.q_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.q_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.q_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.v_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.v_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.16.self_attn.v_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.input_layernorm.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.down_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.down_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.down_proj.weight": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.gate_proj.biases": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.gate_proj.scales": "model-00016-of-00049.safetensors",
+    "transformer.layers.17.mlp.gate_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.mlp.up_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.mlp.up_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.mlp.up_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.post_attention_layernorm.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.k_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.k_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.k_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.o_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.o_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.o_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.q_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.q_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.q_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.v_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.v_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.17.self_attn.v_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.input_layernorm.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.down_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.down_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.down_proj.weight": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.gate_proj.biases": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.gate_proj.scales": "model-00017-of-00049.safetensors",
+    "transformer.layers.18.mlp.gate_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.mlp.up_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.mlp.up_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.mlp.up_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.post_attention_layernorm.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.k_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.k_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.k_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.o_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.o_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.o_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.q_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.q_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.q_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.v_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.v_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.18.self_attn.v_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.input_layernorm.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.down_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.down_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.down_proj.weight": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.gate_proj.biases": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.gate_proj.scales": "model-00018-of-00049.safetensors",
+    "transformer.layers.19.mlp.gate_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.mlp.up_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.mlp.up_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.mlp.up_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.post_attention_layernorm.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.k_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.k_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.k_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.o_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.o_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.o_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.q_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.q_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.q_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.v_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.v_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.19.self_attn.v_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.input_layernorm.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.down_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.down_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.down_proj.weight": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.gate_proj.biases": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.gate_proj.scales": "model-00019-of-00049.safetensors",
+    "transformer.layers.2.mlp.gate_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.mlp.up_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.mlp.up_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.mlp.up_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.post_attention_layernorm.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.k_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.k_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.k_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.o_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.o_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.o_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.q_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.q_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.q_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.v_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.v_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.2.self_attn.v_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.input_layernorm.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.down_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.down_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.down_proj.weight": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.gate_proj.biases": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.gate_proj.scales": "model-00020-of-00049.safetensors",
+    "transformer.layers.20.mlp.gate_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.mlp.up_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.mlp.up_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.mlp.up_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.post_attention_layernorm.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.k_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.k_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.k_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.o_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.o_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.o_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.q_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.q_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.q_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.v_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.v_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.20.self_attn.v_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.input_layernorm.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.down_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.down_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.down_proj.weight": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.gate_proj.biases": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.gate_proj.scales": "model-00021-of-00049.safetensors",
+    "transformer.layers.21.mlp.gate_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.mlp.up_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.mlp.up_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.mlp.up_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.post_attention_layernorm.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.k_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.k_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.k_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.o_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.o_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.o_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.q_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.q_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.q_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.v_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.v_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.21.self_attn.v_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.input_layernorm.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.down_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.down_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.down_proj.weight": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.gate_proj.biases": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.gate_proj.scales": "model-00022-of-00049.safetensors",
+    "transformer.layers.22.mlp.gate_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.mlp.up_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.mlp.up_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.mlp.up_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.post_attention_layernorm.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.k_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.k_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.k_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.o_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.o_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.o_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.q_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.q_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.q_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.v_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.v_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.22.self_attn.v_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.input_layernorm.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.down_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.down_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.down_proj.weight": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.gate_proj.biases": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.gate_proj.scales": "model-00023-of-00049.safetensors",
+    "transformer.layers.23.mlp.gate_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.mlp.up_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.mlp.up_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.mlp.up_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.post_attention_layernorm.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.k_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.k_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.k_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.o_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.o_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.o_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.q_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.q_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.q_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.v_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.v_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.23.self_attn.v_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.input_layernorm.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.down_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.down_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.down_proj.weight": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.gate_proj.biases": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.gate_proj.scales": "model-00024-of-00049.safetensors",
+    "transformer.layers.24.mlp.gate_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.mlp.up_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.mlp.up_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.mlp.up_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.post_attention_layernorm.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.k_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.k_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.k_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.o_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.o_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.o_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.q_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.q_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.q_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.v_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.v_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.24.self_attn.v_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.input_layernorm.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.down_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.down_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.down_proj.weight": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.gate_proj.biases": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.gate_proj.scales": "model-00025-of-00049.safetensors",
+    "transformer.layers.25.mlp.gate_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.mlp.up_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.mlp.up_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.mlp.up_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.post_attention_layernorm.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.k_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.k_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.k_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.o_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.o_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.o_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.q_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.q_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.q_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.v_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.v_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.25.self_attn.v_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.input_layernorm.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.down_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.down_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.down_proj.weight": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.gate_proj.biases": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.gate_proj.scales": "model-00026-of-00049.safetensors",
+    "transformer.layers.26.mlp.gate_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.mlp.up_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.mlp.up_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.mlp.up_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.post_attention_layernorm.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.k_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.k_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.k_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.o_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.o_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.o_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.q_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.q_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.q_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.v_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.v_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.26.self_attn.v_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.input_layernorm.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.down_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.down_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.down_proj.weight": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.gate_proj.biases": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.gate_proj.scales": "model-00027-of-00049.safetensors",
+    "transformer.layers.27.mlp.gate_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.mlp.up_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.mlp.up_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.mlp.up_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.post_attention_layernorm.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.k_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.k_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.k_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.o_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.o_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.o_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.q_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.q_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.q_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.v_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.v_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.27.self_attn.v_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.input_layernorm.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.down_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.down_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.down_proj.weight": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.gate_proj.biases": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.gate_proj.scales": "model-00028-of-00049.safetensors",
+    "transformer.layers.3.mlp.gate_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.mlp.up_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.mlp.up_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.mlp.up_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.post_attention_layernorm.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.k_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.k_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.k_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.o_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.o_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.o_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.q_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.q_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.q_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.v_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.v_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.3.self_attn.v_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.input_layernorm.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.down_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.down_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.down_proj.weight": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.gate_proj.biases": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.gate_proj.scales": "model-00029-of-00049.safetensors",
+    "transformer.layers.4.mlp.gate_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.mlp.up_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.mlp.up_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.mlp.up_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.post_attention_layernorm.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.k_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.k_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.k_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.o_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.o_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.o_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.q_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.q_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.q_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.v_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.v_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.4.self_attn.v_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.input_layernorm.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.down_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.down_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.down_proj.weight": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.gate_proj.biases": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.gate_proj.scales": "model-00030-of-00049.safetensors",
+    "transformer.layers.5.mlp.gate_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.mlp.up_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.mlp.up_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.mlp.up_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.post_attention_layernorm.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.k_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.k_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.k_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.o_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.o_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.o_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.q_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.q_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.q_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.v_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.v_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.5.self_attn.v_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.input_layernorm.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.down_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.down_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.down_proj.weight": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.gate_proj.biases": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.gate_proj.scales": "model-00031-of-00049.safetensors",
+    "transformer.layers.6.mlp.gate_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.mlp.up_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.mlp.up_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.mlp.up_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.post_attention_layernorm.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.k_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.k_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.k_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.o_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.o_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.o_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.q_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.q_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.q_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.v_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.v_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.6.self_attn.v_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.input_layernorm.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.down_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.down_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.down_proj.weight": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.gate_proj.biases": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.gate_proj.scales": "model-00032-of-00049.safetensors",
+    "transformer.layers.7.mlp.gate_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.mlp.up_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.mlp.up_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.mlp.up_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.post_attention_layernorm.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.k_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.k_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.k_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.o_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.o_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.o_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.q_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.q_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.q_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.v_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.v_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.7.self_attn.v_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.input_layernorm.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.down_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.down_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.down_proj.weight": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.gate_proj.biases": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.gate_proj.scales": "model-00033-of-00049.safetensors",
+    "transformer.layers.8.mlp.gate_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.mlp.up_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.mlp.up_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.mlp.up_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.post_attention_layernorm.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.k_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.k_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.k_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.o_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.o_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.o_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.q_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.q_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.q_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.v_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.v_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.8.self_attn.v_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.input_layernorm.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.down_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.down_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.down_proj.weight": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.gate_proj.biases": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.gate_proj.scales": "model-00034-of-00049.safetensors",
+    "transformer.layers.9.mlp.gate_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.mlp.up_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.mlp.up_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.mlp.up_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.post_attention_layernorm.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.k_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.k_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.k_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.o_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.o_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.o_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.q_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.q_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.q_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.v_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.v_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer.layers.9.self_attn.v_proj.weight": "model-00035-of-00049.safetensors",
+    "transformer.norm.weight": "model-00035-of-00049.safetensors",
+    "transformer2.embed_tokens.biases": "model-00035-of-00049.safetensors",
+    "transformer2.embed_tokens.scales": "model-00035-of-00049.safetensors",
+    "transformer2.embed_tokens.weight": "model-00035-of-00049.safetensors",
+    "transformer2.layers.0.input_layernorm.weight": "model-00035-of-00049.safetensors",
+    "transformer2.layers.0.mlp.down_proj.biases": "model-00035-of-00049.safetensors",
+    "transformer2.layers.0.mlp.down_proj.scales": "model-00035-of-00049.safetensors",
+    "transformer2.layers.0.mlp.down_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.gate_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.gate_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.gate_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.up_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.up_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.mlp.up_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.post_attention_layernorm.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.k_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.k_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.k_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.o_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.o_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.o_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.q_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.q_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.q_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.v_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.v_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.0.self_attn.v_proj.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.1.input_layernorm.weight": "model-00036-of-00049.safetensors",
+    "transformer2.layers.1.mlp.down_proj.biases": "model-00036-of-00049.safetensors",
+    "transformer2.layers.1.mlp.down_proj.scales": "model-00036-of-00049.safetensors",
+    "transformer2.layers.1.mlp.down_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.gate_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.gate_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.gate_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.up_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.up_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.mlp.up_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.post_attention_layernorm.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.k_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.k_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.k_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.o_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.o_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.o_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.q_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.q_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.q_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.v_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.v_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.1.self_attn.v_proj.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.10.input_layernorm.weight": "model-00037-of-00049.safetensors",
+    "transformer2.layers.10.mlp.down_proj.biases": "model-00037-of-00049.safetensors",
+    "transformer2.layers.10.mlp.down_proj.scales": "model-00037-of-00049.safetensors",
+    "transformer2.layers.10.mlp.down_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.gate_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.gate_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.gate_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.up_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.up_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.mlp.up_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.post_attention_layernorm.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.k_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.k_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.k_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.o_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.o_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.o_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.q_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.q_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.q_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.v_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.v_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.10.self_attn.v_proj.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.11.input_layernorm.weight": "model-00038-of-00049.safetensors",
+    "transformer2.layers.11.mlp.down_proj.biases": "model-00038-of-00049.safetensors",
+    "transformer2.layers.11.mlp.down_proj.scales": "model-00038-of-00049.safetensors",
+    "transformer2.layers.11.mlp.down_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.gate_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.gate_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.gate_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.up_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.up_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.mlp.up_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.post_attention_layernorm.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.k_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.k_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.k_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.o_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.o_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.o_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.q_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.q_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.q_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.v_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.v_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.11.self_attn.v_proj.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.2.input_layernorm.weight": "model-00039-of-00049.safetensors",
+    "transformer2.layers.2.mlp.down_proj.biases": "model-00039-of-00049.safetensors",
+    "transformer2.layers.2.mlp.down_proj.scales": "model-00039-of-00049.safetensors",
+    "transformer2.layers.2.mlp.down_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.gate_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.gate_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.gate_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.up_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.up_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.mlp.up_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.post_attention_layernorm.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.k_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.k_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.k_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.o_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.o_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.o_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.q_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.q_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.q_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.v_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.v_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.2.self_attn.v_proj.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.3.input_layernorm.weight": "model-00040-of-00049.safetensors",
+    "transformer2.layers.3.mlp.down_proj.biases": "model-00040-of-00049.safetensors",
+    "transformer2.layers.3.mlp.down_proj.scales": "model-00040-of-00049.safetensors",
+    "transformer2.layers.3.mlp.down_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.gate_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.gate_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.gate_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.up_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.up_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.mlp.up_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.post_attention_layernorm.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.k_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.k_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.k_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.o_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.o_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.o_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.q_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.q_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.q_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.v_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.v_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.3.self_attn.v_proj.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.4.input_layernorm.weight": "model-00041-of-00049.safetensors",
+    "transformer2.layers.4.mlp.down_proj.biases": "model-00041-of-00049.safetensors",
+    "transformer2.layers.4.mlp.down_proj.scales": "model-00041-of-00049.safetensors",
+    "transformer2.layers.4.mlp.down_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.gate_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.gate_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.gate_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.up_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.up_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.mlp.up_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.post_attention_layernorm.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.k_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.k_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.k_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.o_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.o_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.o_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.q_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.q_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.q_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.v_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.v_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.4.self_attn.v_proj.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.5.input_layernorm.weight": "model-00042-of-00049.safetensors",
+    "transformer2.layers.5.mlp.down_proj.biases": "model-00042-of-00049.safetensors",
+    "transformer2.layers.5.mlp.down_proj.scales": "model-00042-of-00049.safetensors",
+    "transformer2.layers.5.mlp.down_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.gate_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.gate_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.gate_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.up_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.up_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.mlp.up_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.post_attention_layernorm.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.k_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.k_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.k_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.o_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.o_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.o_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.q_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.q_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.q_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.v_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.v_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.5.self_attn.v_proj.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.6.input_layernorm.weight": "model-00043-of-00049.safetensors",
+    "transformer2.layers.6.mlp.down_proj.biases": "model-00043-of-00049.safetensors",
+    "transformer2.layers.6.mlp.down_proj.scales": "model-00043-of-00049.safetensors",
+    "transformer2.layers.6.mlp.down_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.gate_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.gate_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.gate_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.up_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.up_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.mlp.up_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.post_attention_layernorm.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.k_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.k_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.k_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.o_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.o_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.o_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.q_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.q_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.q_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.v_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.v_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.6.self_attn.v_proj.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.7.input_layernorm.weight": "model-00044-of-00049.safetensors",
+    "transformer2.layers.7.mlp.down_proj.biases": "model-00044-of-00049.safetensors",
+    "transformer2.layers.7.mlp.down_proj.scales": "model-00044-of-00049.safetensors",
+    "transformer2.layers.7.mlp.down_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.gate_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.gate_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.gate_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.up_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.up_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.mlp.up_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.post_attention_layernorm.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.k_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.k_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.k_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.o_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.o_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.o_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.q_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.q_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.q_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.v_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.v_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.7.self_attn.v_proj.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.8.input_layernorm.weight": "model-00045-of-00049.safetensors",
+    "transformer2.layers.8.mlp.down_proj.biases": "model-00045-of-00049.safetensors",
+    "transformer2.layers.8.mlp.down_proj.scales": "model-00045-of-00049.safetensors",
+    "transformer2.layers.8.mlp.down_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.gate_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.gate_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.gate_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.up_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.up_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.mlp.up_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.post_attention_layernorm.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.k_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.k_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.k_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.o_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.o_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.o_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.q_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.q_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.q_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.v_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.v_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.8.self_attn.v_proj.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.9.input_layernorm.weight": "model-00046-of-00049.safetensors",
+    "transformer2.layers.9.mlp.down_proj.biases": "model-00046-of-00049.safetensors",
+    "transformer2.layers.9.mlp.down_proj.scales": "model-00046-of-00049.safetensors",
+    "transformer2.layers.9.mlp.down_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.gate_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.gate_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.gate_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.up_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.up_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.mlp.up_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.post_attention_layernorm.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.k_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.k_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.k_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.o_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.o_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.o_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.q_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.q_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.q_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.v_proj.biases": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.v_proj.scales": "model-00047-of-00049.safetensors",
+    "transformer2.layers.9.self_attn.v_proj.weight": "model-00047-of-00049.safetensors",
+    "transformer2.norm.weight": "model-00047-of-00049.safetensors",
+    "transformer_lm_head.biases": "model-00047-of-00049.safetensors",
+    "transformer_lm_head.scales": "model-00047-of-00049.safetensors",
+    "transformer_lm_head.weight": "model-00048-of-00049.safetensors",
+    "type_info_output_proj.biases": "model-00048-of-00049.safetensors",
+    "type_info_output_proj.scales": "model-00048-of-00049.safetensors",
+    "type_info_output_proj.weight": "model-00049-of-00049.safetensors"
+  }
+}

qwen2_tokenizer/config.json

@@ -0,0 +1,27 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "hidden_act": "silu",
+  "hidden_size": 3584,
+  "initializer_range": 0.02,
+  "intermediate_size": 18944,
+  "max_position_embeddings": 131072,
+  "max_window_layers": 28,
+  "model_type": "qwen2",
+  "num_attention_heads": 28,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 4,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 1000000.0,
+  "sliding_window": 131072,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.37.2",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

qwen2_tokenizer/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": false,
+  "eos_token_id": 151643,
+  "max_new_tokens": 2048,
+  "transformers_version": "4.37.0"
+}

qwen2_tokenizer/tokenizer_config.json

@@ -0,0 +1,40 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": ["<|im_start|>", "<|im_end|>"],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

vocab.yaml

@@ -0,0 +1,13 @@
+- '[verse]'
+- '[chorus]'
+- '[bridge]'
+- '[intro-short]'
+- '[intro-medium]'
+- '[intro-long]'
+- '[outro-short]'
+- '[outro-medium]'
+- '[outro-long]'
+- '[inst-short]'
+- '[inst-medium]'
+- '[inst-long]'
+- '[silence]'