| # 🐟 Fish Speech S2 Pro — Quantization Experiments |
|
|
| **Comprehensive multi-phase quantization of Fish Audio S2 Pro (4.56B params) with voice cloning samples.** |
|
|
| ## Model Architecture |
|
|
| | Component | Layers | Dim | Heads | Params | Size | |
| |-----------|--------|-----|-------|--------|------| |
| | **Slow AR** (LLM backbone) | 36 | 2560 | 32 (GQA, 8 local) | ~4.0B | ~8.5 GB | |
| | **Fast AR** (acoustic decoder) | 4 | 2560 | 32 (GQA, 8 local) | ~0.4B | ~0.8 GB | |
| | **DAC Codec** (RVQ) | — | — | — | — | 1.7 GB | |
| | **Total** | — | — | — | **4.56B** | **~10.8 GB** | |
|
|
| ## Quantization Experiments |
|
|
| ### Phase 1: Proven Approaches (Zero/Near-Zero Quality Loss) |
|
|
| | ID | Method | Target | Expected Size | Compression | Status | |
| |----|--------|--------|--------------|-------------|--------| |
| | **1a** | FP8 (per-row symmetric) | Slow AR | **~6.8 GB** | 1.60x | ✅ Proven ([drbaph/s2-pro-fp8](https://huggingface.co/drbaph/s2-pro-fp8)) | |
| | **1b** | INT4 (group=128) | Slow AR | **~4.8 GB** | 2.24x | ✅ Proven ([baicai1145/s2-pro-w4a16](https://huggingface.co/baicai1145/s2-pro-w4a16)) | |
|
|
| ### Phase 2: Aggressive Approaches (Potential Quality Tradeoffs) |
|
|
| | ID | Method | Target | Expected Size | Compression | Status | |
| |----|--------|--------|--------------|-------------|--------| |
| | **2a** | INT4 (group=128) | All | **~4.9 GB** | 2.19x | 🔬 Experimental | |
| | **2b** | INT8 (per-row) | Slow AR | **~6.8 GB** | 1.60x | ✅ Safe | |
| | **2c** | INT3 (group=128) | Slow AR | **~4.3 GB** | 2.52x | ⚠️ Risky | |
|
|
| ### Phase 3: Extreme Approaches (Quality Degradation Expected) |
|
|
| | ID | Method | Target | Expected Size | Compression | Status | |
| |----|--------|--------|--------------|-------------|--------| |
| | **3a** | INT2 (group=64) | Slow AR | **~3.8 GB** | 2.88x | ❌ Likely degraded | |
| | **3b** | INT2 (group=64) | All | **~3.8 GB** | 2.88x | ❌ Likely degraded | |
|
|
| ## Quick Start |
|
|
| ### Prerequisites |
| - CUDA GPU with ≥24GB VRAM (A100 40/80GB recommended) |
| - Python 3.10+ |
|
|
| ### Run All Phases |
| ```bash |
| # Clone fish-speech and this experiment repo |
| git clone https://github.com/fishaudio/fish-speech.git |
| cd fish-speech-experiments |
| |
| # Install dependencies |
| pip install torch einops loguru ormsgpack hydra-core omegaconf safetensors torchaudio soundfile |
| |
| # Run all phases |
| python scripts/quantize.py --phase all --output ./output |
| |
| # Or run individual phases |
| python scripts/quantize.py --phase 1a # FP8 only |
| python scripts/quantize.py --phase 1b # INT4 only |
| python scripts/quantize.py --phase 2c # INT3 only |
| ``` |
|
|
| ### Upload to Hub |
| ```bash |
| # Requires HF write token |
| huggingface-cli login |
| python scripts/upload_to_hub.py --output ./output |
| ``` |
|
|
| ## Voice Cloning |
|
|
| Each phase generates two audio samples: |
| 1. **`{phase}_tts.wav`** — Text-to-speech without reference |
| 2. **`{phase}_clone.wav`** — Voice cloning from celebrity reference |
|
|
| The reference audio is generated from the base model using a Morgan Freeman-style deep narration: |
| > *"Good morning. I want to tell you something about the universe. Every atom in your body came from a star that exploded. We are all made of star stuff."` |
| |
| ## Existing Quantized Models on HuggingFace |
| |
| | Model | Method | Size | Link | |
| |-------|--------|------|------| |
| | fishaudio/s2-pro | BF16 (original) | 10.8 GB | [Link](https://huggingface.co/fishaudio/s2-pro) | |
| | drbaph/s2-pro-fp8 | FP8 | 6.2 GB | [Link](https://huggingface.co/drbaph/s2-pro-fp8) | |
| | baicai1145/s2-pro-w4a16 | GPTQ INT4 | ~5.5 GB | [Link](https://huggingface.co/baicai1145/s2-pro-w4a16) | |
| | rodrigomt/s2-pro-gguf | GGUF (q2-q8) | 2.4-9.2 GB | [Link](https://huggingface.co/rodrigomt/s2-pro-gguf) | |
| |
| ### GGUF Sizes (from rodrigomt/s2-pro-gguf) |
| | Quant | Size | Notes | |
| |-------|------|-------| |
| | f16 | 9.2 GB | Lossless | |
| | q8_0 | 5.2 GB | Near-lossless | |
| | q6_k | 4.2 GB | Minimal loss | |
| | q5_k_m | 3.8 GB | Slight loss | |
| | q4_k_m | 3.3 GB | Good tradeoff | |
| | q3_k | 2.8 GB | Noticeable loss | |
| | q2_k | 2.4 GB | Significant loss | |
| |
| ## Quantization Details |
| |
| ### FP8 (Phase 1a) |
| - **Method**: Per-row symmetric FP8 (float8_e4m3fn) |
| - **What's quantized**: All `nn.Linear` weights in Slow AR |
| - **What's kept in bf16**: Embeddings, layer norms, Fast AR, codec |
| - **Scale**: Per-row float32 (captures per-channel variation) |
| - **Dequant**: `W_bf16 = W_fp8.to(bfloat16) * scale` |
| - **Quality**: Zero perceptible loss |
|
|
| ### INT4 (Phase 1b) |
| - **Method**: Group-wise symmetric INT4 (group_size=128) |
| - **Range**: [-7, 7] per weight |
| - **Scale**: Per-group float32 |
| - **Target**: Slow AR only (Fast AR + codec in bf16) |
| - **Quality**: Near-zero loss with group_size=128 |
|
|
| ### INT3 (Phase 2c) |
| - **Method**: Group-wise symmetric INT3 (group_size=128) |
| - **Range**: [-3, 3] per weight |
| - **Expected**: Some quality loss, especially on prosody |
| |
| ### INT2 (Phase 3) |
| - **Method**: Group-wise symmetric INT2 (group_size=64) |
| - **Range**: [-1, 0, 1] per weight (ternary!) |
| - **Expected**: Significant quality degradation |
|
|
| ## Files |
|
|
| ``` |
| fish-speech-experiments/ |
| ├── scripts/ |
| │ ├── quantize.py # Main quantization + sample generation script |
| │ ├── run_all_phases.py # Alternative all-in-one script (for HF Jobs) |
| │ └── upload_to_hub.py # Upload results to HuggingFace Hub |
| ├── output/ # Generated quantized models + samples |
| │ ├── samples/ # Audio samples from each phase |
| │ ├── phase1a/ # FP8 quantized model |
| │ ├── phase1b/ # INT4 quantized model |
| │ ├── phase2a/ # INT4 all layers |
| │ ├── phase2b/ # INT8 quantized model |
| │ ├── phase2c/ # INT3 quantized model |
| │ ├── phase3a/ # INT2 quantized model |
| │ ├── phase3b/ # INT2 all layers |
| │ └── all_results.json # Combined results |
| ├── size_analysis.json # Theoretical size analysis |
| └── README.md # This file |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{liao2026fishaudios2technical, |
| title={Fish Audio S2 Technical Report}, |
| author={Shijia Liao and Yuxuan Wang and others}, |
| year={2026}, |
| eprint={2603.08823}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.SD}, |
| } |
| ``` |
|
|
| ## License |
|
|
| Quantized models inherit the [Fish Audio Research License](https://huggingface.co/fishaudio/s2-pro/blob/main/LICENSE.md). |
| Research and non-commercial use only. |
|
|
