Add files using upload-large-folder tool

README.md

@@ -0,0 +1,114 @@
+---
+license: other
+library_name: mlx
+pipeline_tag: text-to-audio
+base_model:
+- tencent/SongGeneration
+tags:
+- mlx
+- apple-silicon
+- music-generation
+- song-generation
+- audiolm
+---
+
+Part of the SongGeneration MLX conversion set.
+
+# SongGeneration-v2-medium-bfloat16
+
+Apple MLX weights for the autoregressive `audiolm` token generator from Tencent SongGeneration v2-medium.
+
+This is not a full-stack pure MLX audio pipeline yet: token generation runs with MLX, while FLAC decoding currently uses the official PyTorch Flow1dVAE / separate-tokenizer bridge in [`ailuntx/SongGeneration-MLX`](https://github.com/ailuntx/SongGeneration-MLX).
+
+## TL;DR
+
+| | |
+|---|---|
+| **Variant** | `v2-medium` |
+| **Precision** | `bfloat16` |
+| **Converted component** | SongGeneration `audiolm` token generator |
+| **Runtime** | [`ailuntx/SongGeneration-MLX`](https://github.com/ailuntx/SongGeneration-MLX) |
+| **Official model** | [`tencent/SongGeneration`](https://huggingface.co/tencent/SongGeneration) |
+| **Official code** | [`tencent-ailab/songgeneration`](https://github.com/tencent-ailab/songgeneration) |
+
+## Quick Start
+
+```bash
+hf download mlx-community/SongGeneration-v2-medium-bfloat16 --local-dir ./models/SongGeneration-v2-medium-bfloat16
+
+git clone https://github.com/ailuntx/SongGeneration-MLX.git
+cd SongGeneration-MLX
+python -m venv .venv
+.venv/bin/pip install -e .
+
+PYTHONPATH=. .venv/bin/python -m songgeneration_mlx.cli \
+  --model ./models/SongGeneration-v2-medium-bfloat16 \
+  --lyrics '[verse] hello from mlx [chorus] sing it again' \
+  --description 'Pop, female vocal, bright production, [Musicality-medium].' \
+  --duration 2 \
+  --top-k 50 \
+  --temperature 0.9 \
+  --output tokens_2s.npz
+```
+
+To decode tokens to FLAC, use the official decoder bridge documented in the GitHub README. The bridge requires the official SongGeneration runtime assets.
+
+## Variants
+
+| Variant | Disk | Notes |
+|---|---:|---|
+| `SongGeneration-v2-medium-fp32` | 10G | high-precision medium baseline |
+| `SongGeneration-v2-medium-bfloat16` | 5.2G | recommended medium quality baseline |
+| `SongGeneration-v2-medium-8bit` | 2.8G | smaller medium checkpoint |
+| `SongGeneration-v2-medium-4bit` | 1.5G | smallest medium checkpoint |
+| `SongGeneration-v2-large-fp32` | 19G | high-precision large baseline |
+| `SongGeneration-v2-large-bfloat16` | 9.5G | large quality baseline |
+| `SongGeneration-v2-large-8bit` | 5.0G | smaller large checkpoint |
+| `SongGeneration-v2-large-4bit` | 2.7G | smallest large checkpoint |
+
+## Layout
+
+```text
+SongGeneration-v2-medium-bfloat16/
+|-- model-00001-of-000xx.safetensors
+|-- model.safetensors.index.json
+|-- config.json
+|-- mlx_manifest.json
+|-- config.official.yaml
+|-- vocab.yaml
+`-- qwen2_tokenizer/
+```
+
+## Validation
+
+Local Apple Silicon validation was run on the medium bfloat16 path:
+
+| Test | Result |
+|---|---|
+| 12s MLX token generation | 550 pattern steps, about 1 minute wall time |
+| 12s official decoder bridge | 73.27s wall time |
+| 12s FLAC | 48kHz stereo, 12.000s, RMS about `0.163` |
+
+The official recent-token repetition penalty is implemented in the MLX runtime. Without it, 12s generations collapse into repeated tokens and decode close to silence.
+
+## License
+
+License follows the upstream SongGeneration release. Check the official model card and repository for the authoritative model license.
+
+## Citation
+
+```bibtex
+@misc{songgeneration-mlx,
+  title  = {SongGeneration-MLX: Apple MLX port of SongGeneration},
+  author = {ailuntx},
+  year   = {2026},
+  url    = {https://github.com/ailuntx/SongGeneration-MLX},
+}
+
+@article{lei2025levo,
+  title   = {LeVo: High-Quality Song Generation with Multi-Preference Alignment},
+  author  = {Lei, Shun and Xu, Yaoxun and Lin, Zhiwei and Zhang, Huaicheng and Tan, Wei and Chen, Hangting and Yu, Jianwei and Zhang, Yixuan and Yang, Chenyu and Zhu, Haina and Wang, Shuai and Wu, Zhiyong and Yu, Dong},
+  journal = {arXiv preprint arXiv:2506.07520},
+  year    = {2025},
+}
+```

config.json

@@ -0,0 +1,129 @@
+{
+  "format": "songgeneration-mlx-audiolm",
+  "source": "tencent/SongGeneration",
+  "official_code": "https://github.com/tencent-ailab/songgeneration",
+  "variant": "v2-medium",
+  "precision": "bfloat16",
+  "runtime": {
+    "hidden_size": 1536,
+    "intermediate_size": 8960,
+    "num_heads": 12,
+    "num_layers": 28,
+    "num_layers_sub": 12,
+    "code_depth": 3,
+    "code_size": 16384,
+    "prompt_len": 10,
+    "frame_rate": 25,
+    "max_position_embeddings": 10000,
+    "max_position_embeddings_sub": 10000,
+    "rope_theta": 500000.0,
+    "rope_theta_sub": 500000.0,
+    "cfg_coef": 1.5
+  },
+  "components": {
+    "audiolm": "converted to MLX safetensors",
+    "qwen_tokenizer": "copied for lyric/style tokenization",
+    "audio_decoder": "use official PyTorch Flow1dVAE/separate tokenizer bridge for now"
+  },
+  "skipped_keys": [
+    "audiolm.out_norm.weight",
+    "audiolm.out_norm.bias",
+    "audiolm.transformer2.lm_head.weight"
+  ],
+  "weight_files": [
+    "model-00001-of-00094.safetensors",
+    "model-00002-of-00094.safetensors",
+    "model-00003-of-00094.safetensors",
+    "model-00004-of-00094.safetensors",
+    "model-00005-of-00094.safetensors",
+    "model-00006-of-00094.safetensors",
+    "model-00007-of-00094.safetensors",
+    "model-00008-of-00094.safetensors",
+    "model-00009-of-00094.safetensors",
+    "model-00010-of-00094.safetensors",
+    "model-00011-of-00094.safetensors",
+    "model-00012-of-00094.safetensors",
+    "model-00013-of-00094.safetensors",
+    "model-00014-of-00094.safetensors",
+    "model-00015-of-00094.safetensors",
+    "model-00016-of-00094.safetensors",
+    "model-00017-of-00094.safetensors",
+    "model-00018-of-00094.safetensors",
+    "model-00019-of-00094.safetensors",
+    "model-00020-of-00094.safetensors",
+    "model-00021-of-00094.safetensors",
+    "model-00022-of-00094.safetensors",
+    "model-00023-of-00094.safetensors",
+    "model-00024-of-00094.safetensors",
+    "model-00025-of-00094.safetensors",
+    "model-00026-of-00094.safetensors",
+    "model-00027-of-00094.safetensors",
+    "model-00028-of-00094.safetensors",
+    "model-00029-of-00094.safetensors",
+    "model-00030-of-00094.safetensors",
+    "model-00031-of-00094.safetensors",
+    "model-00032-of-00094.safetensors",
+    "model-00033-of-00094.safetensors",
+    "model-00034-of-00094.safetensors",
+    "model-00035-of-00094.safetensors",
+    "model-00036-of-00094.safetensors",
+    "model-00037-of-00094.safetensors",
+    "model-00038-of-00094.safetensors",
+    "model-00039-of-00094.safetensors",
+    "model-00040-of-00094.safetensors",
+    "model-00041-of-00094.safetensors",
+    "model-00042-of-00094.safetensors",
+    "model-00043-of-00094.safetensors",
+    "model-00044-of-00094.safetensors",
+    "model-00045-of-00094.safetensors",
+    "model-00046-of-00094.safetensors",
+    "model-00047-of-00094.safetensors",
+    "model-00048-of-00094.safetensors",
+    "model-00049-of-00094.safetensors",
+    "model-00050-of-00094.safetensors",
+    "model-00051-of-00094.safetensors",
+    "model-00052-of-00094.safetensors",
+    "model-00053-of-00094.safetensors",
+    "model-00054-of-00094.safetensors",
+    "model-00055-of-00094.safetensors",
+    "model-00056-of-00094.safetensors",
+    "model-00057-of-00094.safetensors",
+    "model-00058-of-00094.safetensors",
+    "model-00059-of-00094.safetensors",
+    "model-00060-of-00094.safetensors",
+    "model-00061-of-00094.safetensors",
+    "model-00062-of-00094.safetensors",
+    "model-00063-of-00094.safetensors",
+    "model-00064-of-00094.safetensors",
+    "model-00065-of-00094.safetensors",
+    "model-00066-of-00094.safetensors",
+    "model-00067-of-00094.safetensors",
+    "model-00068-of-00094.safetensors",
+    "model-00069-of-00094.safetensors",
+    "model-00070-of-00094.safetensors",
+    "model-00071-of-00094.safetensors",
+    "model-00072-of-00094.safetensors",
+    "model-00073-of-00094.safetensors",
+    "model-00074-of-00094.safetensors",
+    "model-00075-of-00094.safetensors",
+    "model-00076-of-00094.safetensors",
+    "model-00077-of-00094.safetensors",
+    "model-00078-of-00094.safetensors",
+    "model-00079-of-00094.safetensors",
+    "model-00080-of-00094.safetensors",
+    "model-00081-of-00094.safetensors",
+    "model-00082-of-00094.safetensors",
+    "model-00083-of-00094.safetensors",
+    "model-00084-of-00094.safetensors",
+    "model-00085-of-00094.safetensors",
+    "model-00086-of-00094.safetensors",
+    "model-00087-of-00094.safetensors",
+    "model-00088-of-00094.safetensors",
+    "model-00089-of-00094.safetensors",
+    "model-00090-of-00094.safetensors",
+    "model-00091-of-00094.safetensors",
+    "model-00092-of-00094.safetensors",
+    "model-00093-of-00094.safetensors",
+    "model-00094-of-00094.safetensors"
+  ]
+}

config.official.yaml

@@ -0,0 +1,141 @@
+# ================ Train Config ================ #
+lyric_processor:
+max_dur: 270
+min_dur: 30
+prompt_len: 10
+pad_to_max: true
+
+
+# ================ Audio tokenzier ================ #
+audio_tokenizer_checkpoint: Flow1dVAE1rvq_./ckpt/model_1rvq/model_2_fixed.safetensors
+audio_tokenizer_frame_rate: 25
+audio_tokenizer_code_depth: 1
+sample_rate: 48000
+
+audio_tokenizer_checkpoint_sep: Flow1dVAESeparate_./ckpt/model_septoken/model_2.safetensors
+audio_tokenizer_frame_rate_sep: 25
+audio_tokenizer_code_depth_sep: 2
+sample_rate_sep: 48000
+
+# ================ VAE ================ #
+vae_config: ./ckpt/vae/stable_audio_1920_vae.json
+vae_model: ./ckpt/vae/autoencoder_music_1320k.ckpt
+
+# ================== LM =========================== #
+lm:
+  lm_type: Llama # [Llama]
+  dim: 1536
+  intermediate_size: 8960
+  num_heads: 12
+  num_layers: 28
+  num_layers_sub: 12
+  code_depth: 3
+  code_size: 16384
+  max_position_embeddings: 10000
+  max_position_embeddings_sub: 10000
+  rope_theta: 500000.0
+  rope_theta_sub: 500000.0
+  dropout: 0.0
+  use_flash_attn_2: true
+  activation: gelu
+  norm_first: true
+  bias_ff: false
+  bias_attn: false
+  causal: true
+  custom: false
+  memory_efficient: true
+  attention_as_float32: false
+  layer_scale: null
+  positional_embedding: sin
+  xpos: false
+  checkpointing: torch
+  weight_init: gaussian
+  depthwise_init: current
+  zero_bias_init: true
+  norm: layer_norm
+  cross_attention: false
+  qk_layer_norm: false
+  qk_layer_norm_cross: false
+  attention_dropout: null
+  kv_repeat: 1
+
+codebooks_pattern:
+  modeling: delay
+  delay:
+    delays: [ 0, 250, 250 ]
+    flatten_first: 0
+    empty_initial: 0
+
+# ================ Conditioners ===================== #
+classifier_free_guidance:
+  # drop all conditions simultaneously
+  training_dropout: 0.15
+  inference_coef: 1.5
+
+attribute_dropout:
+  # drop each condition separately
+  args:
+    active_on_eval: false
+  text:
+    description: 0.0
+    type_info: 0.2
+  audio:
+    prompt_audio: 0.5
+
+
+use_text_training: True
+fuser:
+  sum: []
+  prepend: [ description, prompt_audio, type_info ] # this order is the SAME with the input concatenation order
+
+conditioners:
+  prompt_audio:
+    model: qt_embedding
+    qt_embedding:
+      code_size: 16384
+      code_depth: 3
+      max_len: ${eval:${prompt_len}*${audio_tokenizer_frame_rate}+2} # 25*10+2+1
+  description:
+    model: QwTokenizer
+    QwTokenizer:
+      token_path: third_party/Qwen2-7B
+      max_len: 600
+      add_token_list: ${load_yaml:conf/vocab.yaml}
+  type_info:
+    model: QwTextTokenizer
+    QwTextTokenizer:
+      token_path: third_party/Qwen2-7B
+      max_len: 100
+
+offload:
+  audiolm:
+    offload_module: self
+    cpu_mem_gb: 0
+    pre_copy_step: 1
+    clean_cache_after_forward: false
+    dtype: torch.float16
+    offload_layer_dict:
+      transformer: 4
+      transformer2: 4
+    ignore_layer_list: []
+    clean_cache_wrapper:
+      module: self
+      method_name: _sample_next_token
+      diff_mem_gb_thre: 2
+    debug: false
+
+  wav_tokenizer_diffusion:
+    offload_module: self.model.model
+    pre_copy_step: 1
+    clean_cache_after_forward: false
+    cpu_mem_gb: -1
+    dtype: null
+    offload_layer_dict:
+      cfm_wrapper: 5
+      hubert: 4
+    ignore_layer_list: []
+    clean_cache_wrapper:
+      module: self.model.model.cfm_wrapper.estimator
+      method_name: forward
+      diff_mem_gb_thre: 1
+    debug: false

mlx_manifest.json

@@ -0,0 +1,129 @@
+{
+  "format": "songgeneration-mlx-audiolm",
+  "source": "tencent/SongGeneration",
+  "official_code": "https://github.com/tencent-ailab/songgeneration",
+  "variant": "v2-medium",
+  "precision": "bfloat16",
+  "runtime": {
+    "hidden_size": 1536,
+    "intermediate_size": 8960,
+    "num_heads": 12,
+    "num_layers": 28,
+    "num_layers_sub": 12,
+    "code_depth": 3,
+    "code_size": 16384,
+    "prompt_len": 10,
+    "frame_rate": 25,
+    "max_position_embeddings": 10000,
+    "max_position_embeddings_sub": 10000,
+    "rope_theta": 500000.0,
+    "rope_theta_sub": 500000.0,
+    "cfg_coef": 1.5
+  },
+  "components": {
+    "audiolm": "converted to MLX safetensors",
+    "qwen_tokenizer": "copied for lyric/style tokenization",
+    "audio_decoder": "use official PyTorch Flow1dVAE/separate tokenizer bridge for now"
+  },
+  "skipped_keys": [
+    "audiolm.out_norm.weight",
+    "audiolm.out_norm.bias",
+    "audiolm.transformer2.lm_head.weight"
+  ],
+  "weight_files": [
+    "model-00001-of-00094.safetensors",
+    "model-00002-of-00094.safetensors",
+    "model-00003-of-00094.safetensors",
+    "model-00004-of-00094.safetensors",
+    "model-00005-of-00094.safetensors",
+    "model-00006-of-00094.safetensors",
+    "model-00007-of-00094.safetensors",
+    "model-00008-of-00094.safetensors",
+    "model-00009-of-00094.safetensors",
+    "model-00010-of-00094.safetensors",
+    "model-00011-of-00094.safetensors",
+    "model-00012-of-00094.safetensors",
+    "model-00013-of-00094.safetensors",
+    "model-00014-of-00094.safetensors",
+    "model-00015-of-00094.safetensors",
+    "model-00016-of-00094.safetensors",
+    "model-00017-of-00094.safetensors",
+    "model-00018-of-00094.safetensors",
+    "model-00019-of-00094.safetensors",
+    "model-00020-of-00094.safetensors",
+    "model-00021-of-00094.safetensors",
+    "model-00022-of-00094.safetensors",
+    "model-00023-of-00094.safetensors",
+    "model-00024-of-00094.safetensors",
+    "model-00025-of-00094.safetensors",
+    "model-00026-of-00094.safetensors",
+    "model-00027-of-00094.safetensors",
+    "model-00028-of-00094.safetensors",
+    "model-00029-of-00094.safetensors",
+    "model-00030-of-00094.safetensors",
+    "model-00031-of-00094.safetensors",
+    "model-00032-of-00094.safetensors",
+    "model-00033-of-00094.safetensors",
+    "model-00034-of-00094.safetensors",
+    "model-00035-of-00094.safetensors",
+    "model-00036-of-00094.safetensors",
+    "model-00037-of-00094.safetensors",
+    "model-00038-of-00094.safetensors",
+    "model-00039-of-00094.safetensors",
+    "model-00040-of-00094.safetensors",
+    "model-00041-of-00094.safetensors",
+    "model-00042-of-00094.safetensors",
+    "model-00043-of-00094.safetensors",
+    "model-00044-of-00094.safetensors",
+    "model-00045-of-00094.safetensors",
+    "model-00046-of-00094.safetensors",
+    "model-00047-of-00094.safetensors",
+    "model-00048-of-00094.safetensors",
+    "model-00049-of-00094.safetensors",
+    "model-00050-of-00094.safetensors",
+    "model-00051-of-00094.safetensors",
+    "model-00052-of-00094.safetensors",
+    "model-00053-of-00094.safetensors",
+    "model-00054-of-00094.safetensors",
+    "model-00055-of-00094.safetensors",
+    "model-00056-of-00094.safetensors",
+    "model-00057-of-00094.safetensors",
+    "model-00058-of-00094.safetensors",
+    "model-00059-of-00094.safetensors",
+    "model-00060-of-00094.safetensors",
+    "model-00061-of-00094.safetensors",
+    "model-00062-of-00094.safetensors",
+    "model-00063-of-00094.safetensors",
+    "model-00064-of-00094.safetensors",
+    "model-00065-of-00094.safetensors",
+    "model-00066-of-00094.safetensors",
+    "model-00067-of-00094.safetensors",
+    "model-00068-of-00094.safetensors",
+    "model-00069-of-00094.safetensors",
+    "model-00070-of-00094.safetensors",
+    "model-00071-of-00094.safetensors",
+    "model-00072-of-00094.safetensors",
+    "model-00073-of-00094.safetensors",
+    "model-00074-of-00094.safetensors",
+    "model-00075-of-00094.safetensors",
+    "model-00076-of-00094.safetensors",
+    "model-00077-of-00094.safetensors",
+    "model-00078-of-00094.safetensors",
+    "model-00079-of-00094.safetensors",
+    "model-00080-of-00094.safetensors",
+    "model-00081-of-00094.safetensors",
+    "model-00082-of-00094.safetensors",
+    "model-00083-of-00094.safetensors",
+    "model-00084-of-00094.safetensors",
+    "model-00085-of-00094.safetensors",
+    "model-00086-of-00094.safetensors",
+    "model-00087-of-00094.safetensors",
+    "model-00088-of-00094.safetensors",
+    "model-00089-of-00094.safetensors",
+    "model-00090-of-00094.safetensors",
+    "model-00091-of-00094.safetensors",
+    "model-00092-of-00094.safetensors",
+    "model-00093-of-00094.safetensors",
+    "model-00094-of-00094.safetensors"
+  ]
+}

model-00002-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cadfb8ae74b6d48e4ebc2e5788220ef00a70618d6ba90aef8b3a0722605d50b
+size 50952384

model-00003-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcb52f514436ea6ffc20dc387de60b4add69f57cec09116601b139d64340ec43
+size 50337888

model-00006-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4ed1114e824b085cc553c9683e170681abb37ca8535d45e5951d9d3a2f9cffb
+size 50334816

model-00007-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9911d98ef55c85f7aa2c3d2718894e483beac43219e4e90c413c76dbcea5e60
+size 64497096

model-00008-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:951917930b5bd938de09c6429eee274e934cfa223b140fe8446e75be83b2733f
+size 50337896

model-00009-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4597fa6e467841a9a6f63c15480161f177cd750c59cd42a1141fcc706c97dc8
+size 50337896

model-00011-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61ad33d538807175ed0832a8282142919417168155cb56b828d326419a2acf12
+size 50338016

model-00014-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e45a51be1539e727ac4a858015c4385add0b303670183b88ac109af5f9a9f78
+size 55050480

model-00015-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c726f524b65d9cc1a07c526ab20168a04f4c36a1a1524d6b113900fcfedfbc4
+size 46406464

model-00022-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42edc0780be172b3f4c84836109be0c9a7ecc2994af0641fdb2f50f2cfbd5562
+size 55050480

model-00023-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf8dd4958d468fe8709ab405b9c07d12b7fd299f71fe485b54ea060ea927cfa8
+size 46406472

model-00026-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74aff76e04fae7b6c1589f2f7c5d3e965537f6c3df892374fa2610f148a6101a
+size 55050480

model-00027-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:146473447f762582cca32cdc055a060dc16cefe8e49624769df2f23e1a208837
+size 46406472

model-00030-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2c8da44adb3faebaeb402d02b498761a86709a75437554febf2eb968a863727
+size 55050480

model-00031-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7d930a844e898a1a78bd3c8a00218a4d9662e0e9da45b4849c9a19c56eda045
+size 46406472

model-00034-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c93107da2de073d647e5275a173b39b3fb9f0a97b5e6c04ba43cf682d024f7a
+size 55050480

model-00035-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ebcbb5b1b0a4fe49fd68bdb2c9f3c5013fae802f3ddd4e1aabc96bdb6a2636a
+size 46406472

model-00042-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:899df93794103faccfb0a821c3b78ca13df222c88e2bf33adfeadda5330ebfba
+size 55050480

model-00043-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07edf0fcfb7538e2cc1a22cfafe2437480e8a6d3bcb7cf04244df3a437aed1ff
+size 46406472

model-00046-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e451c621dd5db114a455e9b6e9dc305a91b5f39a167624badcddae493612b4aa
+size 55050480

model-00047-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf3ec8b6dcfa6e6a6b231d7f97274efbfc59eb9a7602d117559f406f07100683
+size 46406472

model-00048-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c08a9fc1e546eb3434655ba908f4e3b17999087652bc2c22a9cb7ee309be92f
+size 55050480

model-00049-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c34aedf0ed283af53e0554dcd40ed19e289a60ead254c42f845b440cf9b9f630
+size 46406472

model-00050-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c50db817129b34b060e598dbeec29acb0ceb8dd9d7382ea7bd7fe5203829fae
+size 55050480

model-00054-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf4ae42b4b0e70eff41bd77bb66a027d84aa60403ff1b3869628dcc7ed9dd9d5
+size 55050480

model-00055-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f029578a882d2c4914b6055f5449a00e6707b23d93d0d2cfd2eb5e3a0471002b
+size 46406464

model-00062-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ca1d5e3b97e32370e3fae8fdd6092e47323e0ffcc80e41b07ea79e51006469c
+size 55050480

model-00063-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:754629ae6c6443bd892f021eb27705aed39cd354c1185dfc7f2e46aab5ac3944
+size 46406464

model-00066-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f34e5b8e11c61b2a9e47b89dce969b87d8f00a3cb24d545b8beec82c5db888b
+size 55050480

model-00067-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:123a71fd9dba5ecb7fc137f2931875698d1de1bc465cb5b5a372eddc84a977ed
+size 46406440

model-00068-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9684f179ececc1eae638adf53e0f76983f677ad4ec6764c5763497b33f43b4ef
+size 50338016

model-00070-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ed88ac4f04e8dea7bf2cd41479135467f52c45a9a239055f66570abb6417441
+size 46406472

model-00071-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3315fbc19510741c9bd94adcc7df01883804f7be53725cc402a101a687d51b8a
+size 55050480

model-00074-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9110d4f8bae6fe2da7f9ad9ec154e5b1bf09ebaca197b079d6c81d20b0b4b3f1
+size 46406480

model-00075-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06a14191c06c1266b2788e3660083191b57f2b14d63076acfb42e8dfbfc26623
+size 55050480

model-00082-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:969c6b5142eea9711e46df6e194d9c1980fe4681910d739e209727e61b609cd8
+size 46406472

model-00083-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50109b89f3860f3251c1d76df8e2dc0959d2f4f0a9f282f6ce5a7ae99f3dd843
+size 55050480

model-00086-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5dbf6c14cb3dc91039c2a36fe686a77850ff61336325b606444566a64c239ae6
+size 46406472

model-00087-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0272083f55838ce55573dedcc36a95ade5dd1b14bab48e13ad4eb53c138e1a81
+size 55050480

model-00088-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99f143e2f433f409e011f2b2e7d250c7fd9f5768a88e397d99b05a9fd1c6b9ca
+size 46406472

model-00089-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b54530ac37d9bd45ffee6a2a50ac82d6fa68de8efc6352b202e7a6745960cb
+size 55050480

model-00090-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c0d504cfb768681ce8560804a9e3f91d62dd1d10982aa8a7da9905726df95fc
+size 46406472

model-00091-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36c04cf95c9a1275056fe97c410b1d29e4fdf2ca5522c9cc5ac21f4dd6645805
+size 55050480

model-00094-of-00094.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a1e5e0712573d51ee2c96c4c92575ffc76a7a5a5396e6d43e1a2bbaad4957f9
+size 465875056

model.safetensors.index.json

@@ -0,0 +1,390 @@
+{
+  "metadata": {
+    "total_size": 5608833024
+  },
+  "weight_map": {
+    "description_output_proj.weight": "model-00001-of-00094.safetensors",
+    "description_structure_emb.weight": "model-00002-of-00094.safetensors",
+    "emb0.weight": "model-00002-of-00094.safetensors",
+    "layer2_emb0.weight": "model-00003-of-00094.safetensors",
+    "layer2_emb1.weight": "model-00004-of-00094.safetensors",
+    "layer2_emb2.weight": "model-00005-of-00094.safetensors",
+    "linears0.weight": "model-00006-of-00094.safetensors",
+    "linears1.weight": "model-00007-of-00094.safetensors",
+    "mlp.linear0.bias": "model-00007-of-00094.safetensors",
+    "mlp.linear0.weight": "model-00007-of-00094.safetensors",
+    "mlp.linear2.bias": "model-00007-of-00094.safetensors",
+    "mlp.linear2.weight": "model-00007-of-00094.safetensors",
+    "prompt_audio_emb0.weight": "model-00008-of-00094.safetensors",
+    "prompt_audio_emb1.weight": "model-00009-of-00094.safetensors",
+    "prompt_audio_emb2.weight": "model-00010-of-00094.safetensors",
+    "prompt_audio_eot": "model-00010-of-00094.safetensors",
+    "prompt_audio_layer2_eot": "model-00010-of-00094.safetensors",
+    "transformer.embed_tokens.weight": "model-00011-of-00094.safetensors",
+    "transformer.layers.0.input_layernorm.weight": "model-00011-of-00094.safetensors",
+    "transformer.layers.0.mlp.down_proj.weight": "model-00012-of-00094.safetensors",
+    "transformer.layers.0.mlp.gate_proj.weight": "model-00012-of-00094.safetensors",
+    "transformer.layers.0.mlp.up_proj.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.0.post_attention_layernorm.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.0.self_attn.k_proj.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.0.self_attn.o_proj.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.0.self_attn.q_proj.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.0.self_attn.v_proj.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.1.input_layernorm.weight": "model-00013-of-00094.safetensors",
+    "transformer.layers.1.mlp.down_proj.weight": "model-00014-of-00094.safetensors",
+    "transformer.layers.1.mlp.gate_proj.weight": "model-00014-of-00094.safetensors",
+    "transformer.layers.1.mlp.up_proj.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.1.post_attention_layernorm.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.1.self_attn.k_proj.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.1.self_attn.o_proj.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.1.self_attn.q_proj.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.1.self_attn.v_proj.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.10.input_layernorm.weight": "model-00015-of-00094.safetensors",
+    "transformer.layers.10.mlp.down_proj.weight": "model-00016-of-00094.safetensors",
+    "transformer.layers.10.mlp.gate_proj.weight": "model-00016-of-00094.safetensors",
+    "transformer.layers.10.mlp.up_proj.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.10.post_attention_layernorm.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.10.self_attn.k_proj.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.10.self_attn.o_proj.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.10.self_attn.q_proj.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.10.self_attn.v_proj.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.11.input_layernorm.weight": "model-00017-of-00094.safetensors",
+    "transformer.layers.11.mlp.down_proj.weight": "model-00018-of-00094.safetensors",
+    "transformer.layers.11.mlp.gate_proj.weight": "model-00018-of-00094.safetensors",
+    "transformer.layers.11.mlp.up_proj.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.11.post_attention_layernorm.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.11.self_attn.k_proj.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.11.self_attn.o_proj.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.11.self_attn.q_proj.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.11.self_attn.v_proj.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.12.input_layernorm.weight": "model-00019-of-00094.safetensors",
+    "transformer.layers.12.mlp.down_proj.weight": "model-00020-of-00094.safetensors",
+    "transformer.layers.12.mlp.gate_proj.weight": "model-00020-of-00094.safetensors",
+    "transformer.layers.12.mlp.up_proj.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.12.post_attention_layernorm.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.12.self_attn.k_proj.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.12.self_attn.o_proj.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.12.self_attn.q_proj.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.12.self_attn.v_proj.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.13.input_layernorm.weight": "model-00021-of-00094.safetensors",
+    "transformer.layers.13.mlp.down_proj.weight": "model-00022-of-00094.safetensors",
+    "transformer.layers.13.mlp.gate_proj.weight": "model-00022-of-00094.safetensors",
+    "transformer.layers.13.mlp.up_proj.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.13.post_attention_layernorm.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.13.self_attn.k_proj.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.13.self_attn.o_proj.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.13.self_attn.q_proj.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.13.self_attn.v_proj.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.14.input_layernorm.weight": "model-00023-of-00094.safetensors",
+    "transformer.layers.14.mlp.down_proj.weight": "model-00024-of-00094.safetensors",
+    "transformer.layers.14.mlp.gate_proj.weight": "model-00024-of-00094.safetensors",
+    "transformer.layers.14.mlp.up_proj.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.14.post_attention_layernorm.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.14.self_attn.k_proj.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.14.self_attn.o_proj.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.14.self_attn.q_proj.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.14.self_attn.v_proj.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.15.input_layernorm.weight": "model-00025-of-00094.safetensors",
+    "transformer.layers.15.mlp.down_proj.weight": "model-00026-of-00094.safetensors",
+    "transformer.layers.15.mlp.gate_proj.weight": "model-00026-of-00094.safetensors",
+    "transformer.layers.15.mlp.up_proj.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.15.post_attention_layernorm.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.15.self_attn.k_proj.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.15.self_attn.o_proj.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.15.self_attn.q_proj.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.15.self_attn.v_proj.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.16.input_layernorm.weight": "model-00027-of-00094.safetensors",
+    "transformer.layers.16.mlp.down_proj.weight": "model-00028-of-00094.safetensors",
+    "transformer.layers.16.mlp.gate_proj.weight": "model-00028-of-00094.safetensors",
+    "transformer.layers.16.mlp.up_proj.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.16.post_attention_layernorm.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.16.self_attn.k_proj.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.16.self_attn.o_proj.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.16.self_attn.q_proj.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.16.self_attn.v_proj.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.17.input_layernorm.weight": "model-00029-of-00094.safetensors",
+    "transformer.layers.17.mlp.down_proj.weight": "model-00030-of-00094.safetensors",
+    "transformer.layers.17.mlp.gate_proj.weight": "model-00030-of-00094.safetensors",
+    "transformer.layers.17.mlp.up_proj.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.17.post_attention_layernorm.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.17.self_attn.k_proj.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.17.self_attn.o_proj.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.17.self_attn.q_proj.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.17.self_attn.v_proj.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.18.input_layernorm.weight": "model-00031-of-00094.safetensors",
+    "transformer.layers.18.mlp.down_proj.weight": "model-00032-of-00094.safetensors",
+    "transformer.layers.18.mlp.gate_proj.weight": "model-00032-of-00094.safetensors",
+    "transformer.layers.18.mlp.up_proj.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.18.post_attention_layernorm.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.18.self_attn.k_proj.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.18.self_attn.o_proj.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.18.self_attn.q_proj.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.18.self_attn.v_proj.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.19.input_layernorm.weight": "model-00033-of-00094.safetensors",
+    "transformer.layers.19.mlp.down_proj.weight": "model-00034-of-00094.safetensors",
+    "transformer.layers.19.mlp.gate_proj.weight": "model-00034-of-00094.safetensors",
+    "transformer.layers.19.mlp.up_proj.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.19.post_attention_layernorm.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.19.self_attn.k_proj.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.19.self_attn.o_proj.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.19.self_attn.q_proj.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.19.self_attn.v_proj.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.2.input_layernorm.weight": "model-00035-of-00094.safetensors",
+    "transformer.layers.2.mlp.down_proj.weight": "model-00036-of-00094.safetensors",
+    "transformer.layers.2.mlp.gate_proj.weight": "model-00036-of-00094.safetensors",
+    "transformer.layers.2.mlp.up_proj.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.2.post_attention_layernorm.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.2.self_attn.k_proj.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.2.self_attn.o_proj.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.2.self_attn.q_proj.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.2.self_attn.v_proj.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.20.input_layernorm.weight": "model-00037-of-00094.safetensors",
+    "transformer.layers.20.mlp.down_proj.weight": "model-00038-of-00094.safetensors",
+    "transformer.layers.20.mlp.gate_proj.weight": "model-00038-of-00094.safetensors",
+    "transformer.layers.20.mlp.up_proj.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.20.post_attention_layernorm.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.20.self_attn.k_proj.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.20.self_attn.o_proj.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.20.self_attn.q_proj.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.20.self_attn.v_proj.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.21.input_layernorm.weight": "model-00039-of-00094.safetensors",
+    "transformer.layers.21.mlp.down_proj.weight": "model-00040-of-00094.safetensors",
+    "transformer.layers.21.mlp.gate_proj.weight": "model-00040-of-00094.safetensors",
+    "transformer.layers.21.mlp.up_proj.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.21.post_attention_layernorm.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.21.self_attn.k_proj.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.21.self_attn.o_proj.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.21.self_attn.q_proj.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.21.self_attn.v_proj.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.22.input_layernorm.weight": "model-00041-of-00094.safetensors",
+    "transformer.layers.22.mlp.down_proj.weight": "model-00042-of-00094.safetensors",
+    "transformer.layers.22.mlp.gate_proj.weight": "model-00042-of-00094.safetensors",
+    "transformer.layers.22.mlp.up_proj.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.22.post_attention_layernorm.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.22.self_attn.k_proj.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.22.self_attn.o_proj.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.22.self_attn.q_proj.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.22.self_attn.v_proj.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.23.input_layernorm.weight": "model-00043-of-00094.safetensors",
+    "transformer.layers.23.mlp.down_proj.weight": "model-00044-of-00094.safetensors",
+    "transformer.layers.23.mlp.gate_proj.weight": "model-00044-of-00094.safetensors",
+    "transformer.layers.23.mlp.up_proj.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.23.post_attention_layernorm.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.23.self_attn.k_proj.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.23.self_attn.o_proj.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.23.self_attn.q_proj.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.23.self_attn.v_proj.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.24.input_layernorm.weight": "model-00045-of-00094.safetensors",
+    "transformer.layers.24.mlp.down_proj.weight": "model-00046-of-00094.safetensors",
+    "transformer.layers.24.mlp.gate_proj.weight": "model-00046-of-00094.safetensors",
+    "transformer.layers.24.mlp.up_proj.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.24.post_attention_layernorm.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.24.self_attn.k_proj.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.24.self_attn.o_proj.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.24.self_attn.q_proj.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.24.self_attn.v_proj.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.25.input_layernorm.weight": "model-00047-of-00094.safetensors",
+    "transformer.layers.25.mlp.down_proj.weight": "model-00048-of-00094.safetensors",
+    "transformer.layers.25.mlp.gate_proj.weight": "model-00048-of-00094.safetensors",
+    "transformer.layers.25.mlp.up_proj.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.25.post_attention_layernorm.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.25.self_attn.k_proj.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.25.self_attn.o_proj.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.25.self_attn.q_proj.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.25.self_attn.v_proj.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.26.input_layernorm.weight": "model-00049-of-00094.safetensors",
+    "transformer.layers.26.mlp.down_proj.weight": "model-00050-of-00094.safetensors",
+    "transformer.layers.26.mlp.gate_proj.weight": "model-00050-of-00094.safetensors",
+    "transformer.layers.26.mlp.up_proj.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.26.post_attention_layernorm.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.26.self_attn.k_proj.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.26.self_attn.o_proj.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.26.self_attn.q_proj.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.26.self_attn.v_proj.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.27.input_layernorm.weight": "model-00051-of-00094.safetensors",
+    "transformer.layers.27.mlp.down_proj.weight": "model-00052-of-00094.safetensors",
+    "transformer.layers.27.mlp.gate_proj.weight": "model-00052-of-00094.safetensors",
+    "transformer.layers.27.mlp.up_proj.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.27.post_attention_layernorm.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.27.self_attn.k_proj.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.27.self_attn.o_proj.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.27.self_attn.q_proj.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.27.self_attn.v_proj.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.3.input_layernorm.weight": "model-00053-of-00094.safetensors",
+    "transformer.layers.3.mlp.down_proj.weight": "model-00054-of-00094.safetensors",
+    "transformer.layers.3.mlp.gate_proj.weight": "model-00054-of-00094.safetensors",
+    "transformer.layers.3.mlp.up_proj.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.3.post_attention_layernorm.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.3.self_attn.k_proj.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.3.self_attn.o_proj.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.3.self_attn.q_proj.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.3.self_attn.v_proj.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.4.input_layernorm.weight": "model-00055-of-00094.safetensors",
+    "transformer.layers.4.mlp.down_proj.weight": "model-00056-of-00094.safetensors",
+    "transformer.layers.4.mlp.gate_proj.weight": "model-00056-of-00094.safetensors",
+    "transformer.layers.4.mlp.up_proj.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.4.post_attention_layernorm.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.4.self_attn.k_proj.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.4.self_attn.o_proj.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.4.self_attn.q_proj.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.4.self_attn.v_proj.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.5.input_layernorm.weight": "model-00057-of-00094.safetensors",
+    "transformer.layers.5.mlp.down_proj.weight": "model-00058-of-00094.safetensors",
+    "transformer.layers.5.mlp.gate_proj.weight": "model-00058-of-00094.safetensors",
+    "transformer.layers.5.mlp.up_proj.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.5.post_attention_layernorm.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.5.self_attn.k_proj.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.5.self_attn.o_proj.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.5.self_attn.q_proj.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.5.self_attn.v_proj.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.6.input_layernorm.weight": "model-00059-of-00094.safetensors",
+    "transformer.layers.6.mlp.down_proj.weight": "model-00060-of-00094.safetensors",
+    "transformer.layers.6.mlp.gate_proj.weight": "model-00060-of-00094.safetensors",
+    "transformer.layers.6.mlp.up_proj.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.6.post_attention_layernorm.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.6.self_attn.k_proj.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.6.self_attn.o_proj.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.6.self_attn.q_proj.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.6.self_attn.v_proj.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.7.input_layernorm.weight": "model-00061-of-00094.safetensors",
+    "transformer.layers.7.mlp.down_proj.weight": "model-00062-of-00094.safetensors",
+    "transformer.layers.7.mlp.gate_proj.weight": "model-00062-of-00094.safetensors",
+    "transformer.layers.7.mlp.up_proj.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.7.post_attention_layernorm.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.7.self_attn.k_proj.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.7.self_attn.o_proj.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.7.self_attn.q_proj.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.7.self_attn.v_proj.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.8.input_layernorm.weight": "model-00063-of-00094.safetensors",
+    "transformer.layers.8.mlp.down_proj.weight": "model-00064-of-00094.safetensors",
+    "transformer.layers.8.mlp.gate_proj.weight": "model-00064-of-00094.safetensors",
+    "transformer.layers.8.mlp.up_proj.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.8.post_attention_layernorm.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.8.self_attn.k_proj.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.8.self_attn.o_proj.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.8.self_attn.q_proj.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.8.self_attn.v_proj.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.9.input_layernorm.weight": "model-00065-of-00094.safetensors",
+    "transformer.layers.9.mlp.down_proj.weight": "model-00066-of-00094.safetensors",
+    "transformer.layers.9.mlp.gate_proj.weight": "model-00066-of-00094.safetensors",
+    "transformer.layers.9.mlp.up_proj.weight": "model-00067-of-00094.safetensors",
+    "transformer.layers.9.post_attention_layernorm.weight": "model-00067-of-00094.safetensors",
+    "transformer.layers.9.self_attn.k_proj.weight": "model-00067-of-00094.safetensors",
+    "transformer.layers.9.self_attn.o_proj.weight": "model-00067-of-00094.safetensors",
+    "transformer.layers.9.self_attn.q_proj.weight": "model-00067-of-00094.safetensors",
+    "transformer.layers.9.self_attn.v_proj.weight": "model-00067-of-00094.safetensors",
+    "transformer.norm.weight": "model-00067-of-00094.safetensors",
+    "transformer2.embed_tokens.weight": "model-00068-of-00094.safetensors",
+    "transformer2.layers.0.input_layernorm.weight": "model-00068-of-00094.safetensors",
+    "transformer2.layers.0.mlp.down_proj.weight": "model-00069-of-00094.safetensors",
+    "transformer2.layers.0.mlp.gate_proj.weight": "model-00069-of-00094.safetensors",
+    "transformer2.layers.0.mlp.up_proj.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.0.post_attention_layernorm.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.0.self_attn.k_proj.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.0.self_attn.o_proj.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.0.self_attn.q_proj.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.0.self_attn.v_proj.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.1.input_layernorm.weight": "model-00070-of-00094.safetensors",
+    "transformer2.layers.1.mlp.down_proj.weight": "model-00071-of-00094.safetensors",
+    "transformer2.layers.1.mlp.gate_proj.weight": "model-00071-of-00094.safetensors",
+    "transformer2.layers.1.mlp.up_proj.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.1.post_attention_layernorm.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.1.self_attn.k_proj.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.1.self_attn.o_proj.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.1.self_attn.q_proj.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.1.self_attn.v_proj.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.10.input_layernorm.weight": "model-00072-of-00094.safetensors",
+    "transformer2.layers.10.mlp.down_proj.weight": "model-00073-of-00094.safetensors",
+    "transformer2.layers.10.mlp.gate_proj.weight": "model-00073-of-00094.safetensors",
+    "transformer2.layers.10.mlp.up_proj.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.10.post_attention_layernorm.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.10.self_attn.k_proj.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.10.self_attn.o_proj.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.10.self_attn.q_proj.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.10.self_attn.v_proj.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.11.input_layernorm.weight": "model-00074-of-00094.safetensors",
+    "transformer2.layers.11.mlp.down_proj.weight": "model-00075-of-00094.safetensors",
+    "transformer2.layers.11.mlp.gate_proj.weight": "model-00075-of-00094.safetensors",
+    "transformer2.layers.11.mlp.up_proj.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.11.post_attention_layernorm.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.11.self_attn.k_proj.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.11.self_attn.o_proj.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.11.self_attn.q_proj.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.11.self_attn.v_proj.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.2.input_layernorm.weight": "model-00076-of-00094.safetensors",
+    "transformer2.layers.2.mlp.down_proj.weight": "model-00077-of-00094.safetensors",
+    "transformer2.layers.2.mlp.gate_proj.weight": "model-00077-of-00094.safetensors",
+    "transformer2.layers.2.mlp.up_proj.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.2.post_attention_layernorm.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.2.self_attn.k_proj.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.2.self_attn.o_proj.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.2.self_attn.q_proj.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.2.self_attn.v_proj.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.3.input_layernorm.weight": "model-00078-of-00094.safetensors",
+    "transformer2.layers.3.mlp.down_proj.weight": "model-00079-of-00094.safetensors",
+    "transformer2.layers.3.mlp.gate_proj.weight": "model-00079-of-00094.safetensors",
+    "transformer2.layers.3.mlp.up_proj.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.3.post_attention_layernorm.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.3.self_attn.k_proj.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.3.self_attn.o_proj.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.3.self_attn.q_proj.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.3.self_attn.v_proj.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.4.input_layernorm.weight": "model-00080-of-00094.safetensors",
+    "transformer2.layers.4.mlp.down_proj.weight": "model-00081-of-00094.safetensors",
+    "transformer2.layers.4.mlp.gate_proj.weight": "model-00081-of-00094.safetensors",
+    "transformer2.layers.4.mlp.up_proj.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.4.post_attention_layernorm.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.4.self_attn.k_proj.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.4.self_attn.o_proj.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.4.self_attn.q_proj.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.4.self_attn.v_proj.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.5.input_layernorm.weight": "model-00082-of-00094.safetensors",
+    "transformer2.layers.5.mlp.down_proj.weight": "model-00083-of-00094.safetensors",
+    "transformer2.layers.5.mlp.gate_proj.weight": "model-00083-of-00094.safetensors",
+    "transformer2.layers.5.mlp.up_proj.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.5.post_attention_layernorm.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.5.self_attn.k_proj.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.5.self_attn.o_proj.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.5.self_attn.q_proj.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.5.self_attn.v_proj.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.6.input_layernorm.weight": "model-00084-of-00094.safetensors",
+    "transformer2.layers.6.mlp.down_proj.weight": "model-00085-of-00094.safetensors",
+    "transformer2.layers.6.mlp.gate_proj.weight": "model-00085-of-00094.safetensors",
+    "transformer2.layers.6.mlp.up_proj.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.6.post_attention_layernorm.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.6.self_attn.k_proj.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.6.self_attn.o_proj.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.6.self_attn.q_proj.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.6.self_attn.v_proj.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.7.input_layernorm.weight": "model-00086-of-00094.safetensors",
+    "transformer2.layers.7.mlp.down_proj.weight": "model-00087-of-00094.safetensors",
+    "transformer2.layers.7.mlp.gate_proj.weight": "model-00087-of-00094.safetensors",
+    "transformer2.layers.7.mlp.up_proj.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.7.post_attention_layernorm.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.7.self_attn.k_proj.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.7.self_attn.o_proj.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.7.self_attn.q_proj.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.7.self_attn.v_proj.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.8.input_layernorm.weight": "model-00088-of-00094.safetensors",
+    "transformer2.layers.8.mlp.down_proj.weight": "model-00089-of-00094.safetensors",
+    "transformer2.layers.8.mlp.gate_proj.weight": "model-00089-of-00094.safetensors",
+    "transformer2.layers.8.mlp.up_proj.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.8.post_attention_layernorm.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.8.self_attn.k_proj.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.8.self_attn.o_proj.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.8.self_attn.q_proj.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.8.self_attn.v_proj.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.9.input_layernorm.weight": "model-00090-of-00094.safetensors",
+    "transformer2.layers.9.mlp.down_proj.weight": "model-00091-of-00094.safetensors",
+    "transformer2.layers.9.mlp.gate_proj.weight": "model-00091-of-00094.safetensors",
+    "transformer2.layers.9.mlp.up_proj.weight": "model-00092-of-00094.safetensors",
+    "transformer2.layers.9.post_attention_layernorm.weight": "model-00092-of-00094.safetensors",
+    "transformer2.layers.9.self_attn.k_proj.weight": "model-00092-of-00094.safetensors",
+    "transformer2.layers.9.self_attn.o_proj.weight": "model-00092-of-00094.safetensors",
+    "transformer2.layers.9.self_attn.q_proj.weight": "model-00092-of-00094.safetensors",
+    "transformer2.layers.9.self_attn.v_proj.weight": "model-00092-of-00094.safetensors",
+    "transformer2.norm.weight": "model-00092-of-00094.safetensors",
+    "transformer_lm_head.weight": "model-00093-of-00094.safetensors",
+    "type_info_output_proj.weight": "model-00094-of-00094.safetensors"
+  }
+}

vocab.yaml

@@ -0,0 +1,13 @@
+- '[verse]'
+- '[chorus]'
+- '[bridge]'
+- '[intro-short]'
+- '[intro-medium]'
+- '[intro-long]'
+- '[outro-short]'
+- '[outro-medium]'
+- '[outro-long]'
+- '[inst-short]'
+- '[inst-medium]'
+- '[inst-long]'
+- '[silence]'